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organization of sentences in text were identified for further study (Years 2 & 3). Four 
types of connectors were examined: sequentials, adversati ves , additives, and causals. 
The effects on recall of including explicit sequence connectors in text were similar for 
native English and ESL speakers: inclusion led to greater recall. The markers affected 
the reading behavior of the native English but not the ESL speakers. Analyses of the 
traces of the reading behavior revealed that most individuals used at least two or 
three global strategies for reading. Localized rereading strategies were related 
to both structural and semantic properties of the texts. A second series of studies 
examined knowledge of connectors as cohesion devices. A multiple choice, cloze task 
in conjunction with response justifications and confidence ratings indicated that 
native English speakers were more often correct than ESL speakers but the differential 
pattern of difficulty across the connectors was identical across the language groups. 
When causal or additive connectors were the appropriate choice, performance was better 
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of the functions and meanings of the four types of connectors. Error analyses 
indicated that there was a general bias toward selecting additive and causal alterna- 
tives, especially among the least proficient ESL students. Response justifications 
for incorrect responses indicated that a major source of error was inaccurate 
comprehension of the relation indicated by the text. When incorrect relational 
inferences were made, students had difficulty selecting the connector alternative 
that matched the inferred (but incorrect) relation. 

Overall, project activities indicate that ESL and native English speakers attempt to 
use similar strategies when they read text. Most individuals in each group demonstrated 
flexibility in the strategies they used for comprehension. Differences in performance 
appeared to reside in the likelihood that the strategy produced the "correct" result 
and in processing efficiency. Important questions for subsequent research are those 
that address (1) the nature of the cognitive, motivational and attitudinal costs of 
inefficient processing, and (2) the developmental course of efficient processing in 
English by ESL speakers. 
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Reasoning and Comprehension Processes of LInflulstIc Minority Persons 

Learning from Text 
The general goals of the three year project "Reasoning and Comprehonsion of 
Linguistic Minority Persons learning from Text" have been to examine the processes and 
proficiencies called upon when nonnative speakers of English attempt to learn new 
information from written text. A particular focus has been on text and task characteristics 
that interact with learner skills, leading in some cases to highly successful performance but 
in others to quite marginal performance. A specific concern was to identify the nature of the 
problems, if any, that are unique to nonnative English speakers. The empirical activities 
conducted under the auspices of this project include: 

• A verbal protocol investigation of the strategies students used to answer questions 
from texts typical of the introductory textbook material prevalent in first and second year 
college-level courses (Goldman & Dur^n, 1987; 1988) 

• An analysis of the rhetorical and stylistic mechanisms authors of text use to 
provide cues to both global and local organization of the presented information (Durdn, 
Goldman & Smith, 1989). 

• A series of experimental studies of the effects of enumeration markers on the 
reading time and recall behavior of native and nonnative English speaking college students 
(Goldman, 1988a,b,c). 

• A series of experimental studies of native and nonnative English speaking college 
students' understandings of the functions and appropriate usage of fo'jr types of 
conjunctions, a category of cohesion devices that specify the logical relationships among 
informational units in text (Goldman & Murray, 1989). 

Several Macintosh software applications were developed to conduct the 
experimental studies. Each program presents text and allows the tracking of the reading 
behavior of the students. One application, READiTl . presents text segments in sequential 
order and permits the student to go back and forth among segments (Saul, Pohi, & Goldman, 
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1988). Inspection tinrje data are collected and there is an application thai summarizes the 
data for each passage. The second application is a variant of the moving window technique 
{Gontier, Saul, & Goldman, 1989). An entire text is presented but the actual text is masked 
with black lines. Students select the sentence they want to read and only thas sentence is 
exposed. Sequence and inspection time are recorded. Thase applications have permitted the 
analysis and comparison at reading strategies employed by native and nonnative English 
speakers as they perform the experimental tasks. 

This final report summarizes the results of the foregoing project activities. The 
overarching theoretical framework and assumptions about comprehension and reasoning that 
have guided all of the work are reviewed first. This section is followed by a disi^usslon o'i the 
specific investigations and the major results of each. Further inform,ation regarding each 
project is available in the indicated technical reports and publications. 

THEORETICAL FRAMfSVypRK OF THg PROJf^pT 

The theoretical framework that has guided this work is that reasoning and 
comprehension are the processes involved in acquiring new information from written text. 
We assume that comprehension is an interactive process; as such, the characteristics of the 
learner, the text, and the task impact the cognitive and metacognitive activities associated 
with learning (e.g.. Brown, Campione, & Day, 1981). A text may be said to be understood 
when a coherent representation of It has been constructed by the reader (e.g., Kintsch & van 
Dijk, 1978). The ease with which such a representation can be constructed is affected by 
text factors such as vocabulary, sentence structures, and local and global organizational 
devices. It is also affected by how much the reader already knows about the topic of the text. 
Finally, the purpose for reading and the specific task thsx. the learner is trying to 
accomplish influence the specific sort of representation that will be most functional. 

Comprehension difficulties often arise because the level of linguistic proficency 
and the degree of domain-specific knowledge assumed by text exceed that possessed by the 
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reader. For oxample, unfamiliar vocabulary may make a text difficult to understand. One 
purpose of informational text is to communicate the meaning of key concepts and terms in 
the domain. Thus, text whose function is to impart such new information is likely to have 
mariy unfamiliar vocabulary torms. In addition, comprehension difficulty may be 
associated with understanding how the words in a sentence and/or the sentences in the entire 
passage relate to one another. The strategies readers use for resolving comprehension 
difticulties often vary but can include ignoring words that are not understood, rereading, and 
consulting an outside expert (Collins & Smith, 1981). 

Empirical investigations of comprehension as an interactive process involving the 
learner, ihe task anc» the m<i are scarce (see for examples of such work Ammon, 1987; 
Langsf, 1985). in our project activities we have been concerned with two learner 
characteristics: (1) the language proficiency of nonnative English speakers, and (2) 
domaln-fipecific prior knowledge. We have focused on language proficiency particularly as 
It ruUtes to tha language used in academic texts. Prior knowledge Is of interest particularly 
as it interacts with language proficiency. High versus low knowledge effects on 
comprehension and reasoning have been demonstrated by a number of researchers (e.g., 
Chiesi, Spillich, & Voss, 1979; Dee Lucas & Larkin, 1986; Spilich, Vesonder, Chiesi. & 
Voss, 1979). However, little is known about the relationship between general language 
proficiencies and domain-specific knowledge as they affect the learner's task of acquiring 
new information from text. 

In our work we have focused on text charscteristics that involve language 
structures that express relationships among individual units of information. Such devices 
impact text cohesion and the learner's ability io construct a coherent representation of the 
text. Some of the relevant language structures are most evident at the sentence level. These 
include, e.g., conjunctions, conditionals, performatives, and quantification terns and 
phrases (see for further discussion Celce-Murcia & Larson-Freeman, 1983; Halliday & 



ERIC 



7 



Has'in, 1976). Other relevant language structures operate at the level of an entire text and 
provide an organizing relationship for all of the ideas in the text. Examples of such global 
structures are compare/con ttast, thesis/evidence, and cause-effect . These global 
structures are typically signalled by rhetorical devices at the paragraph level and 
awareness of them allows the learner to more easily encode and construct a coherent mental 
representation of the incoming information (e. g. Brewer, 1980; Meyer, Brandt, & Bluth, 
1980). 

The task characteristics that we have been looking at arise from a consideration of 
comprehension skills hierarchies (Bormuth, Manning, Carr & Pearson, 1970; Rosenshine. 
1980) and envisionment levels (Fillmore, 1983; Kay, 1987; Langer, 1986). 
Comprehension tasks can be thought of as varying along a continuum reflecting the degree to 
which the task can be successfully completed with "only" the text as compared to requiring 
material and knowledge external to the text (see for discussion Goldman, 1985). 
Comprehension skills hierarchies reflect this continuum in that these ordered skills imply 
an increasingly more sophisticated understanding as one moves from literal- 
comprehension of the text, to making simple inferences from the text, to engaging in more 
complex inferential reasoning based on the information in the text. Variations in 
dependency on the text are also reflected in the envisionment levels of reasoning proposed by 
Fillmore (1983). Envisionment levels refer to variations in individuals' understanding of 
the world described in a text. These levels range from the most basic, i.e., "Understanding 
independent statements in a text", to the most complex, i.e., "Embellishing the text world in 
light of existing knowledge and in terms of' possible extensions and underlying generalities". 
The more basic envisionment levels are more dependent on the text, reflect more literal 
understanding of the material and assessment typically involves different kinds of 
con.prehension tasks than does assessment of more complex understandings. 
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Project activities have examined a range of envisionment leveis using question 
answering (Goidman & Durdn, 1988), cued recail (Goidman, 1988a,b), and cioze 
completion tasks (Goldman & Murray, 1989). Furthermore we have coiiected severai 
dependent measures that permitted inferences regarding metacognitive and strategic aspects 
of performance (Goldman, 1988c). These measures Included confidence ratings, reading 
time, verbal justifications, and a method for tracking the sequence in which the sentences in 
a text are examined. 

II. ANSWERING QUESTIONS FROM OCEANOGRAPHY TEXTS 

We began our investigations of the Interaction of learner, task and text 
characteristics with a descriptive study of behavior of students engaged in answering 
questions based on material they had read in their textbooks, a task typical of introductory- 
level college courses. Participants were native-English and ESL speakers who varied in 
their levels of expertise In oceanography and oceanography-related domains. Task variables 
were manipulated by including questions for which the answers required students to engage 
In different "levels" of comprehension. In fact, interactions between text and task 
characteristics created situations where questions that looked literal" actually required 
going beyond the Information in the text. For example, if the text stated the answer to a yes- 
no question, the question is categorized as a literal question. On the other hand, the answer 
to this same question might not be explicitly stated in the text. In which case the iearnor 
would have to engage In inferential reasoning to answer it. in the latter case, correctly 
answering would indicate more complex levels of comprehension than in the former case. 
Furthermore, different question-answering strategies would be expected in the two cases. 

Thus, depending on the nature of the text, the task, and the relationship between the 
text and the task, different strategies for accomplishing the task may be called for. Some 
questions and tasks may be answered in an almost "rote" fashion by, literally, matching the 
words in the question to the same words in the text. Other questions and tasks may require 



using the Information in the text to draw conclusions and/or relate the material to the 
learner's own world (Goldman, 1985; Langer, 19C5; Rosenshine, 1980). In general, the 
difficulty associated with achieving correct performance was expected to increase as 
"distance" /rom the text increased. 

Our purpose in the research was to examine the strategies learners employed, 
given their levels of expertise in the domain, their English language proficiency and the 
relationship between the particular question and the text. For this purpose, we employed a 
think-aloud protocol methodology. Despite the potential difficulty of talking and working 
with English text concurrently, we note that think-aloud procedures have been efficaciously 
used with ESL students in previous work on reading and reading strategies (e.g. Benedetto, 
1986; Block, 1986 a,b). Within task vat ation was introduced by including questions that 
varied along the text-dependency contin jum and hence had the potential for eliciting 
different strategies from the students. 

Method 

Materials 

Two sections of text were selected from the book the subjects were using in their 
Introductory oceanography course, Qfifian SfiifiDfifi (Stowe, 1983). The text sections were 
comparable in terms of length (approximately 1500 words in each selection) and featured a 
representative sampling of the types of concepts students in the course have to deal with, 
including definitions of ierms, properties of geophysical phenomena, mathematical relations 
and physical laws. (For further details of the materials, refer to Goldman & Duran, 1987, 
1 988.) 

Comprehension questions were largely drawn from the study guide developed by the 
instructor of the course, but in three cases were developed by the authors. Correct answers 
to each question were provided by the course instructor. Specific relau'ons between the 
questions and the texts were determined by identifying and comparing the explicit predicate 
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propositions and coherence relations occurring in each. Basad on this analysis we 
determined that questions varied in terms of how closely they matched the surface structure 
of the text; in their reliance on technical k.iowledge (especially vocabulary), presumably 
acquired prior to taking the oceanography course; and in their dependence on reasoning 
processes for correct solution. This variation in types of questions permitted the 
examination of different solution methods, and their relation to the various question types. 

Question types 

Five question types were identified based on an analysis of (1) the relationship 
between the question and text, (2) the demands made on the knowledge base, and (3) 
previous empirical work on differences between various types of questions (e.g., Bormuth, 
et al., 1970; Goldman, ^985; Langer, 1986; Rosenshine, 1980). Table 1 summarizes the 
five types and indicates how the 18 separate questions employed in the study were 
distributed across types. In the first type of question a verbatim relationship exists 
between the question and the text: The answer Is given explicitly and there Is a direct match 
between the question wording and the text wording. To answer this kind of question, the 
learner merely has to locate the appropriate section in the text and find the matching 
language. 

Insert Table 1 about here 

The second type of question involves a paraphrase relationship between the 
question and the text: The text explicitly gives the answer if certain vocabulary 
equivalences and conversions are made by the learner. These conversions frequently depend 
on prior knowledge and apparently assume that the learner already knows certain technical 
vocabulary (e.g. factor, ratio) because such terms are not defined anywhere in the text. 
This vocabulary issue may be extremely cogent for predicting nonnative English speakers' 
success on these types of questions. For this type of question, once the appropriate 
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vocabulary conversions are done and conceptual equivalences established, the relevant text 
portion typically is circumscribed and localized in one small area of the text. Thus, 
question types 1 and 2 tend to involve locating specific sentences in the text and reflect 
largely literal comprehension and local envisionment. 

The third type of question, verbatim look-up plus comparison, Involves reasoning 
with infcrmation found in the text. The text gives the necessary information explicitly and 
there is usually a direct match between the question wording and the text. Thus, Jocating the 
information proceeds much like in type 1 questions. However, once the information is 
''found^ it must then be compared to other information. The fourth type of question 
requires integration of information across several paragraphs of the text. There may bo 
either a verbatim or paraphrase relationship between the question and the text but the text 
provides the relevant information in a number of paragraphs. The information must be 
coordinated and analyzed to construct the correct answer. These types of questions would 
lend themselves to partially correct answers if learners were to locate only one relevant 
section of text. This type of question thus requires integrating local knowledge. 

The last type of question requires reasoning, application and/or computation. 
Questions of this type involve using a text-provided formula, rule or relationship to get the 
precisely correct answer. Locating the formula, rule or relationship involves a verbatim 
or paraphrase match or look-up process but answeis to questions of this type cannot be 
found in the book directly. Rather, the learner must disembed the relevant information and 
apply It to a new situation described in the question. This types of question thus requires 
envisionment or comprehension levels that involve extending the text beyond its own 
confines whereas the other four types of questions stay within the boundaries of the text. 
(The texts, questions and answers are available in Goldman & Dur^n, 1987) 
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Seven students volunteered to participate in the question-answering protocol 
study. Six of the students were enrolled in the introductory oceanography course on campus 
and the seventh (HS) was the teaching assistant for th^ course and our "a priori" expert in 
the content domain. Students completed a background questionnaire dealing with academic 
information, language skills- and study habits. Table 2 provides a summary of the most 
pertinent information from this questionnaire. The teaching assistant and three of the 
students were native English speakers (MR, LH, and DW). The other three students (GL, II, 
and EH) were from three different nonEngll!i.h language backgrounds but were relatively 
proficient In English. GL rated his English language skills "extremely good"; II and EH each 
rated their skills "good". The four native English speakers rated their own English language 
skills "extremely good." The language of instruction during high school had been English for 
all the students. EH had had the least exposure to English, having entered the United States 
six years ago, at which time she had her first contacts with English. II and GL were first 
exposed to English at the age of 4 years. Of the nonnative English speakers, only EH reported 
using her native language daily. GL and II reported that they used their native languages to 
read newspaper or magazine material, but rarely. 



Insert Table 2 about here 

All students' high-school backgrounds included courses in chemistry and/or 
biology. MR and DW were college seniors, GL a junior, LH a sophomore and EH and II were 
freshmen at the time of the study and thus experience with college-level science courses 
varied. MR had an extensive background in physics and HS in oceanography. GL had taken 
astronomy. The other three students had no college-level science courses but I! had had 
physics In high school. Based on this background information, it appeared that DW, LH, EH 
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and perhaps II would have less prior knowledge relevant to questions on oceanography than 
the other three. Table 2 also describes the self-ratings of English language skills related to 
academic performance. These ratings suggest that the native and nonnatlve English speakers 
differed primarily in terms of their command of science vocabulary with the latter 
indicating only moderate command of such terms. Ratings on learning from English lectures 
were "good- and "extremely good" for all students. Learning from English text skills were 
rated higher ("extremely good") by the native English speakers as compared to the 
nonnative English speakers ("good"). 
Procedure. 

Volunteers were recruited from students In the introductory oceanography class. 
They were paid $5 per hour for their participation. Students were told that we were 
interested In how they went about answering questions on oceanography matei iai and that we 
wanted them to think-aloud as they worked on several questions that we would give them. 
Then the think-aloud method was described and modeled, following procedures outlined by 
Ericsson and Simon (1984). 

Following practice with the verbal protocol procedure, the first text selection was 
presented and subjects were free to read over it. Then the questions were presented one at a 
time for the students to answer. Interviewer probes were used to (1) encourage subjects to 
report what they were looking at and thinking about as they answered each question, and (2) 
clarify and disambiguate some of the subjects' commr.ua. Approximately 3-4 weeks later, 
students returned and they worked with the second text selection. After it was read over, the 
questions were presented one at a time and subjects were asked to answer them. Sessions 
were audiotaped and later transcribed. 



14 



Analysis of the Question-Answering Protocols 
Conceptual Process model 

We developed a general conceptual model of the processes and sources of 
information that might be involved in answering questions based on academic text materials. 
This conceptual model, shown in Table 3, indicates four major processing events and the 
goals associated with each one. In addition, metacognitive processing events are shown as 
optional. Monitoring can apply to any of the processing events and/or to learners' general 
thoughts about their performance, the task, the text, etc. In discussing the model, examples 
from the protocols are used illustratively. However, the conceptual model was not derived 
Itflin the protocols; rather its development was influenced by prior empirical research 
illustrating that comprehension ar^d question answering may be profitably conceptualized as 
constructive, problem solving activities (e.g., Ammon, 1987; Bransford & Johnson, 1973; 
Collins, Brown & LaTkIn, 1980; Goldman, 1985; McLaughlin, 1987). We were 
particularly interested in the relation between the question and the text and how that 
relation impacted, if at all, on students' solution strategies. 



Insert Table 3 about here 

Each major processing event shown in Table 0 may be further "unpacked," or 
expanded, into its constituent processes, goals and procedures. An extsnded discussion of the 
conceptual model appears in Goldman and Dur^n (1988) and only a summary presentation Is 
presented here. The first processing event, question encoding, will be used to illustrate the 
nature of the model. Question encoding has two primary goals. The first goal Is to determine 
the type of answer required by the question. Essentially, the learner needs to have some 
sense of (1) what form the answer will take, i.e., one word, a sentonce, several sentences; 
(2) the structural requirements on the answer, i.e., two concepts must be compared versus 



discussing a single concept; (3) whetl-ier verbai or numeric information is sought; and (4) 
the llkeiihood that the answer was already given in the text or is present In memory, i.e., 
does the question wording indicate that a novei application or situation is being addressed? 
Without even considering the relation between the question and the text, differences in these 
four areas imply that different strategies are likely to be useful and that arriving at the 
answers involves differential degrees of cognitive effort. 

Furthermore, as we discussed in the presentation of question types, questions vary 
In terms of (1) the level of envisionment demanded ( Langer, 1985), (2) the relationship 
between the text and the question (Pearson & Johnson, 1978; Rosenshine, 1S80) and, (3) 
the type and extent of reasoning necessary to arrive at the answer (Goldman, 1985). 
Variations In the reasoning requirements of questions are not coterminus with the explicit- 
Implicit factor, nor with the envisionment level. They depend on the form of the target 
answer.' Some questions involve simply reporting a specific "fact" or datum; others require 
explanation, comparison and sometimes application to a new situation. 

Two sample questions illustrate the nature of the differences among the questions 
and kinds of answers demanded. In the example question below there are several parts, each 
of which requires a numeric response, but which differ in terms of how that numeric 
response may be determined: 

a) Suppose air at 25" C is saturated (100% relative humidity). What fraction of the air is 
water? (b) What would be the answer if the temperature were 35*? (c) 15"? (d) 5^? 
(e) Does the answer change by roughly a factor of 2 for every 10<*? 

The answers to parts a and e can be found verbatim in the text; the answers to c and d can 
be read directly from a table given in the text. The answer to b must be computed but it can 
be computed in two ways. The individual can either apply the rule referred to in part e of 
tho question or can interpolate from the series given in the table in the text. A second 
example question Illustrates an explanation question where vocabulary is a critiual issue; 
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"Briefly explain how 'echoes' can be used to measure the depths of discontinuities in the 

earth's internal structure." To locate relevant portions of the text or to access the 

appropriate concept in memory, the learner must understand the equivalence between the 

phrase in the question, depths of discontinuities , and the phrase used in the text interfaces 

between materials. Furthermore, there are several sections of the text that are relevant 

and two factors that need to be discussed: time and speed. Time Is discussed in a three 

sentence section that is separated by nine sentences from the section that deals explicitly 

with the time and speed relationship. To give the complete answer, the learner must 

integrate across sections of text and extract the pertinent relationship. 

"The first item (time) could be read from your seismograph, but there is no direct way to 
l<now the speed of seismic waves deep within the earth. Fortunately, this information may 
be inferred from data, due to the other important property of waves. 

This second property is...." (speed is mentioned again 3 sentences later.) 

The text does indicate that the (earner ought to read on for the complete answer but 
sensitivity to this rhetorical signal may be related to English language proficiency. 

To accomplish the goal of determining what tvpe of reasoning and what type of 
answer the question requires, the primary means is to rely on the language of the question. 
Analysis of the language draws on prior experiences with other academic texts and a 
sensitivity to the semantics of various question words, such as how many , what and why . To 
the knowledgeable individual, these questions words provide cues to the appropriate form of 
the answer. For example, a "how many" question ought to indicate the need for a specific 
quantity, a "why" for a causal or logical explanation of the phenomenon mentioned in the 
question. In addition, learners may use the task context and constraints to further define 
the nature of the answer. In an untlmed situation, one can afford to be more discursive than 
in a time limited one. Furthermore, the space provided for the answer often gives the 
learner Information regarding the extent of the answer, e.g., a blank in the middle o a 
sentence versus a question followed by a half page of blank space. 
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The second major goal of question encoding is to determine starting points for a 
search space. A primary means of doing this is to use technical terms ar^d keywords 
mentioned in the question as entry points to memory and/or the textbook. When the question 
uses words that match those used In the headings and subheadings of the text, learners are 
virtually assured of a reasonably well-defined search space in the text. Keyword matching 
is also facilitated by the use of boldface or italics in the body of the text. In defining a search 
space in memory, the keywords in the question behave similarly but the success of a match 
will vary depending on learners' individual mental representations of the text information. 
Regardless of whether the learner is defining a search space In the text o.' in semantic 
memory, there is the possibility that the words in the question will be seductively appealing 
as cues, when in fact the question really requires original thinking or the application of 
presented material. Questions of this sort are not only at more complex envisionment 
levels but also may provide "false" signals to the learner, by creating the impression that 
the answer Is "in the book". Thus, question encoding has important outcomes for the Search 
component of the question-answering model (see Table 3) In that question analysis helps 
define the search space as well as the type of answer for which the learner is searching. 

The Search processing events are "unpacked 'n a manner similar to question 
encoding. Search may proceed either in memory or in an external source such as a 
textbook, notes or supplementary reference material and the goals differ somewhat for 
memory compared to external searches. The goals for memory search distinguish between 
two outcomes. In the first goal, the answer is found in memory and no external search is 
undertaken. If memory learch is abandoned for oxiernal-source search, a second goal of the 
memory search is to determine alternate sources for the answer. Thus, a memory search 
may not produce the answer but may further define, and refine, subsequent search spaces. 
External source search can thus be facilitated by a memory search that does not yield the 
specific answer to the question. The memory search phase of the model is the one about 
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which we have the least to say based on the protocol data (see Reder, 1987, for a recent 
theoretical and empirical focus on question answering from memory). 

Goals for external searches are similar to those for memory In that we assume the 
operation of decision, monitoring and evaluation processes during the course of the search. 
In searching a textbook, learners may have three major goals that are interdependent and 
interrelated. One goal is to delimit the search space by using the results of question 
encoding and memory search to constrain and guide textbook processing. This goal becomes 
increasingly important as the amount of potentially appropriate text material increases. A 
second goal, finding information relevant to the question, will be achieved with less effort if 
the search space can be appropriately limited but will be more difficult if the search space 
is incorrectly constrained. 

Given a section of text to search, search may be global or guided. Global searches 
begin with the learner having only a vaguely defined search space, e. g., "I'll look in the text 
for that. It must be there somewhere." Such searches are typically characterized by 
scanning or skimming and may be exhaustive or self-terminating. Monitoring, evaluation 
and decision processes are tuned to the occurrence of concepts, vocabulary, or other text 
material that "matches" the requirements of the question. When a match is encountered, the 
learner's attention becomes focused on that section of text and more careful examination of 
the text replaces the skimming behavior. The search process, in effect, changes to a guided 
search. Guided search is, from the outset, targeted at a defined search space, a localized area 
of the text. Keywords and topics mentioned in the question are used by the learner to 
explicitly identify and focus on specific sections of text, e. g., "This question is about 
gravity. That's section two. I'll look there." 

The third goal becomes active once question-relevant Information is located: the 
information must be meaningfully processed in the context of the task defined by the 
question. The learner must extract the relevant information in a form that suits the task 
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demands. For some questions, just recognizing tlie right materlai and reading it from tlie 
text Is sufficient. For other types of questions, summarization may be caiied for and in stiii 
others, the learner may need to engage in extended reasoning and knowledge application 
processes. 

A transcribed natural language protocol for one of the subjects, DW, is shown in 
Table 4 and illustrates the interdependence and interreiatedness of the goals and processing 
events in the question encoding, memory search and text search components of the model. 
This pailicular question required the explanation of an everyday occurrence: "Explain why 
you feel cold when you get out of the shower^ Although the text explicitly stated the answer 
in one self-contained paragraph, text section B.2S15-18, references to the scientific 
concept involved, evaporation, occur in the three paragraphs preceding, and in the four 
paragraphs following, the paragraph containing the answer. DW first encoded the question 
by reading It, defined the topic as one of the major ones in that section of text, latent heat," 
and then reread the question (lines 1-3). She then gave an answer from memory, "because 
of the evaporation process", but evaluated her answer as insufficient for the question (line 
5). She proceeded to do a text search (line 6), presumably to gather more information on 
evaporation and why the process works that way. She began reading in the appropriate 
section, B.2 Latent heat, but two paragraphs below the one containing the answer (lines 7- 
11). She skipped up the page to the topic sentence of the paragraph containing the answer 
(B.2S15). She then stated an answer that paraphrased 815 in nontechnical terms but was 
only partially correct because her answer deleted the details of molecular movement and its 
relationship to evaporation and temperature (lines 13-15), details necessary to the 
explanation. She evaluated this answer as missing something (line 16) and adopted a global 
search strategy of skimming from the beginning of Chapter 12 (lines 17-21) until she got 
to the first sentence of section B.I, which uses the Keyword "evaporate" (line 22) She read 
the next two sentences (lines ?3-25) and then skimmed the remainder of section B.I, 
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assumedly because the "evaporation" match didn't lead anywhere on the shower explanation 
(lines 26-37). She then read the first six sentences under section B.2 (lines 38-44), and 
recognized this as the relevant material (line 45). She re-read part of it (S3 - S6 in lines 
46-50) and then continued reading about perspiration and eva[)oration (lines 51, 52). DW 
then proceeded to paraphrase the infornnation she had just read (lines 53-61). However, 
she got tangled up, reread S8 and then reverted to her original answer. "Well, I guess I'd 
say evaporation, and leave it at that." 

Insert Table 4 about here 

DW started by defining the search space as information on latent hoat; she then 
retrieved an answer from memory, evaporation. This answer guided her attempts to search 
the text for more information. However, DW apparently failed to see the Importance of 
continuing with the paragraph starting with S15 (line 12); instead she en-jaged in a global 
text search that took her away from the relevant information. She never got back to this 
paragraph but attempted to conclude her answer to this question with a paraphrase of a less- 
relevant text section (B.2S7 & 8). However this section is the first section under her self- 
defined topic, "latent heat," that mentioned "evaporation." She monitored her understanding 
and the paraphrase as not terribly direct but instead of resuming a search process, she 
retreated back to the same answer she had originally retrieved from memory and the one 
that provided the impetus for the text search in the first place. We can only speculate on 
why she "gave up" at this point. 

DW's protocol illustrates the fourth component of the question answering model, 
Construct and output an answer (see Table 3). Two interrelated goals involved in this 
process are (1) answer the question completely and (2) match the type of answer to the 
type of question. DW was answering a question that required an explanation and she 
indicated by her self-question "But what is it that causes that?" that her initial answer 



21 



("evaporation") did not really qualify as an explanation. Anotlier learner, GL, indicated his 
attempt to meet the goal of answering completely, after having given a correct, partial 
answer to one of the questions. He stated: "Ok. I'll read it again and to find out exactly what 
these are used for. 1 mean I know they're used to measure depth but I know they have other 
(uses)." An optional goal in the answer component is to demonstrate that the information 
has been Incorporated into the knowledge base. Several of the students repeatedly 
paraphrased text sections they had just read aloud. We interpreted this behavior as their 
efforts to comply with this optional goal. 

A final processing event in the question-answering model is optional but 
encompasses processes related to the role of metacognitive behavior in the question 
answering process. At any time, and for any of the processing events, confirmation, 
monitoring and evaluation may be invoked by the learner. The outcomes of such monitoring, 
In part, determine the sequencing and Interplay between the various processing events. For 
example, after reading a iengthy section of the text, a number of students were obsen/ed to 
reread the question. There are several related explanations for why that particular 
sequence occurred: first, the learner might have monitored memory for the specifics of the 
question and determined that the trace was not sufficiently active; second, the juxtaposition 
of the text and the question might help in the determination of the appropriateness of the 
just-read text to the exact question. Reder (1987) has suggested a third possibility: 
rereading may increase the familiarity of the concepts and thus the likelihood of success for 
direct retrieval of the answer. 
Solution stratRgifi?^, 

Based on the conceptual model of question answering we derived a typology of 
solufion strategies by considering which processing events, goals within processing events, 
and monitoring processes were involved in solution. A specific solution strategy consists of 
question encoding, some type of search, some set of additional processing activities 
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(including the empty set) and output of the answer. Four types of search were specified, 
each of which could be accompanied by any of four "additional" processing activities: 
question analysis, reasoning/inference, process monitoring or product monitoring. 

The four search types were Memory Search (A), Text Search (B), Both memory 
and text search (C), and Text Search, Reason Beyond (D). Each search type could be 
augmented by the occurrence of one or more of four additional processing activities: 
Question analysis refers to rereading or analyzing the nature of the question. Reasoning and 
inference refer to efforts to logically manipulate given or remembered information. 
Process and product monitoring refer to metacognitive behavior directed at the progress of 
any cognitive activity (process monitoring) or at the results of search and retrieval 
processes (product monitoring). 
Protocol Scoring . 

Each subject's transcribed protocol for each question was analyzed to determine 
which of the solution strategies was reflected in the empirical data. To do this, we coded the 
natural language descriptions of the protocols for cognitive actions (e.g., recall, read, 
compare, monitor, evaluate) and the types of information on which the action operated (e.g., 
text material, question, own knowledge). Table 5 shows the coded version of the natural 
language protocol that was given in Table 4. 

Insert Table 5 about here 

Based on the coded protocol, a solution strategy for each question answered by each 
subject was determined. We adopted the convention of a 5-place code for designating 
solution strategies. The first place indicated the type of search (A - D) and the remaining 
four indicated the presence (1) or absence (0) of each of the processing activities described 
above. Furthermore, a superscript gives the evaluation of the final answer produced by the 
learner, with 1 indicating correct, 2 indicating qualitatively correct (but not 
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quantitatively) for those questions wliere a specific numerical value was requested, 3 
Indicating a partially correct answer, 4 an incorrect answer, and 5 no answer given. 
Applying this procedure to DW's coded protocol (Table 6) leads to the designation C.101l3. 
The search type is "C because DW recalled an answer from memory and also consulted the 
textbool< prior to producing her final answer. Additional processing activities were 
rereading the question (question analysis) process monitoring ("I know that there's 
something in the book that I'm missing") and product monitoring ("Why does the process 
work that way?"). She is credited with a partially correct answer because she never 
explained tiosi her answer, evaporation, led to feeling cold, although she made two 
explanatory attempts. Note that the 5-place code for the solution strategies does not 
distinguish between single and multiple occurrences of processing events, i.e., if one section 
of text was read it was counted the same as if several sections were read. 

The results of applying our scoring/coding scheme to each subject's solution for 
each question was a distribution of solution strategies across learners and types of questions. 
These data were the basis of the subsequent analyses. 

Results and Discussion 

Several important trends were reflected in the solution strategy data. Consistent 
with an interactive, constructive comprehension model, task characteristics and learner 
characteristics affected the nature of the solution strategies. Task characteristics were 
reflected in the type of question: the various types were differentially difficult and there 
was evidence of the use of different solution strategies depending on the type of question. 
Learner characteristics were reflected in level of expertise within the domain and in 
language proficiency. While individuals tended to be relatively consistent in their strategic 
approach to the task, individual differences were observed and were related to both domain 
expertise and language background. The trends associated with task and with learner are 
discussed below. 
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Question type anri task diffiniiltv 

There was a predicted order of difficulty for the five types of questions, with Type 
1 expected to be the easiest and Type 5 the most difficult. Types 2, 3, and 4 were predicted 
to be roughly equivalent in difficulty. We also expncted, based on the notion that the need for 
various processing activities depends on the relation between the question and the text, that 
solution strategies would reflect variations in task demands created by question factors (see 
also Reder, 1987, Experiment 6). Specifically, reasoning/inference and question analysis 
were predicted to occur less often for Type 1 questions as compared to the other types. 
Finally, we predicted that monitoring processes would occur more often for questions 
requiring the coordination of information from multi|5le paragraphs, e.g., Types 3, 4, and 5. 

Question type difficulty was measured by the percent correct (quantitative or 
qualitative) answers and Is shown in the first row of Table 6. As predicted, Type 1 
questions were the easiest: 89% of the answers were correct. Type 4 questions were also 
relatively easy, 71% norrect. The unexpected ease with which these were solved Is probably 
due to the verbatim relation that held between this sample of Type 4 questions and the text. 
In this partic jiar task context the difference between Types 1 and 4 was the amount of text 
that matched the question, with Type 4 questions matching over longer segments thus 
requiring integration over several sections of the text. 

Insert Table 6 about here 

Those question. types requiring reasoning were successfully solved 62% (Type 3) 
and 57% (Type 5) of the time, whereas Type 2 questions were the most difficult, showing 
only 39% successful solutions. We had expected that this type of question might be more 
difficult for ESL than for native English speakers because of the vocabulary conversions 
req-iired. Both groups had equivalent correct rates (42% for the ESL and 38% for the 
native English speakers). However, the two groups differed in that the native English 
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speakers were more likely to have partially correct answer (31%) than the ESL speakers, 
none of whom got partial credit on answers to questions of this type. Thus, the most difficult 
question type. Type 2, was the type requiring knowledge of vocabulary and conceptual 
equivalence, primarily for technical, natural science and oceanograp^ly terms. 
Question type and solution strategies 

Consistent with our expectations, learners engaged In different solution strategies 
depending on the type of question they were attempting to answer. As shown In Table 6, foi 
the Type 1 questions, which have a verbatim relation to the text, 90% of the correct 
solutions Involved a simple "search plus retrieve answer" strategy. Of these, 58% were 
text searches and 32% were memory searches. Furthermore, about half of these single 
source searches involved no other processing activities. Correct solutions to Type 4 
questions, which also had verbatim relations to the text, were simple "search plus retrieve 
answer" strategies 85% of the time, but about half of these were text and half m.emory 
searches. Not surprisingly reasoning and inference processes rarely occurred for either 
Type 1 or 4 questions. Solutions to the two types of verbatim questions differed principally 
In terms of the more frequent use of monitoring In the Type 4 questions. 

Type 3 questions, verbatim "look up" plus comparison, were similar to Types 1 
and 4 in terms of the searcn strategies used in correct solutions (85% search single 
source). Type 3 differed from Types 1 and 4 in that 54% of the correct solutions employed 
reasoning and inference processes, a difference consistent with the nature of these questions. 
Type 3 differed from Type 4 but was similar to Type 1 in that less than half of the correct 
solutions (46%) featured monitoring, principally checking the product or answer before 
stopping. Thus, for these three types of questions that all In some measure involve 
verbatim relations between text and question, there is a comparable degree of reliance on 
text and memory search leading to correct answer retrieval. 
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Correct solutions to Type 5 questions, requiring use of a formuia or rule given in 
the text, are distinguished from Types 1 , 4 and 3 in terms of the appearance of the 
disconfirming search strategy. I.e., the search leads to the realization that the answer is not 
"given" explicitly In the text but must be computed (search Type D). In conjunction with 
this search type, process and product monitoring are very frequent in the correct solutions 
and are more frequent than in the three preceding types of questions. Similar to Type 3, 
reasoning and inference occurred In 63% of the correct solutions. Solutions to Type 5 
questions usually featured additional processes, a marked difference from Types 1 and 4. 

Correct solutions to Typo 2 questions were a cross between Types 1 and 4 solution 
strategies. Like Type 1 , 83% of the correct solutions involved searching one source and 
answering; like Type 4, there was a high degree of monitoring (75% of the solutions). 
Note however, lhat these data represent only 39% of the attempted solutions. In other 
words, for those questions correctly answered, the successful vocabulary conversion 
allowed learners to treat these questions like verbatim questions. Given the high failure 
rate on Type 2 questions (61%), consideration of the solution strategies for both correct 
and incorrect final answers is informative. Over all solutions, the most frequent search 
strategy was a text search (17 of 28 solutions); however, 59% (10) of these failed to lead 
to correct solution. Another five memory search solutions (63% of 8 memory searches 
attempted) failed to lead to a correct solution. Two of three solutions using both sources 
failed. The inability to map the language of the question onto the language of the text appears 
to have been the critical obstacle to successful solution. The fact that the native English 
speakers were more likely than the ESL to be at least partially correct on this question type 
supports the conclusion that difficulties on this type of question may be related to English 
language proficiency. 

Our findings indicate that questions requiring more literal and explicitly 
presented text Information are easier to answer than those roquiring more global levels of 
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envlslonment. We confirmed Langer's (1986) finding that monitoring (Langer's equivalent 
is "puzzling") was more frequent for the questions with higher error rates. In the present 
study, these were the questions Involving application of the Information or vocabulary 
conversions. These types of questions draw most heavily on English langage proficiency and 
domain-related prior knowledge. We next consider the relation between subject-matter 
expertise and the solution strategies of individual learners. 
Learner characteristics and solution stra tegy patterns 

The learners in this study differed in terms of expertise in the specific subject- 
matter domain, oceanography, and in terms of oceanography-related course work. The 
"experts" were HS, who was the teaching assistant for the oceanography course, and MR, 
who had an extensive hard-science (physics) background. One of the nonnatlve English 
speakers, GL, had taken a related science course (astronomy) and we dubbed him "high" 
knowledge. A second nonnatlve English speaker, II, had taker, physics In high school whereas 
the remaining nonnatlve English (EH) speaker and the two native English speakers (DW and 
LH) had had no science courses relevant to oceanography. The characteristics of the solution 
strategies for each of these Individuals are discussed in order of "expertise" and compared in 
terms of the quantifiable characteristics, including percent correct, given in Table 7. 

Insert Table 7 about here 

The observed differences among the protocols can be summarized in the following 
way: The "expert" native English learners tended to rely on memory searches; the more 
successful learner engaged in optional answer confirmation. Of the lower knowledge native 
English speakers, one tended to rely on memory (LH) and had a high rate of incorrect 
answers. When she did search the text she found the correct answers only for Type 1 
(verbatim) questions. Finally, DW relied on the text to a greater degree than the other 
native English speakers. Her solution strategies were extensive and quite often involved 
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question analysis, monitoring and reasoning, in generai, tiie nonnative Engiisii speal<ers 
reiied on the text to a greater degree than the native Engiish speakers. The highest 
l<nowiedge and mcrA successfui nonnative Engiish speal<er, GL, reiied aimost exciusiveiy on 
the texi and engaged in frequent question analysis and monitoring activities. EH was iess 
successfui in her efforts to use the text and was hampered frequently by 
vocabulary/conceptual gaps. Finally, II used memory and text searches but was frequently 
hampered by faulty or inadequate question encoding or text interpretation. 

The foregoing characterizations were based on the simple presence or absence of 
the various processing activities. A complementary analysis considered the frequency of 
various processing activities. The frequency data index the length of solution. Two 
measures wore computed from the coded protocols: number of actions (read, reread, skim, 
etc.) summed over all 18 questions answered by each learner, and number of metacognitive 
actions (monitor, evaluate, etc.) also summed over all 18 questions answered by each 
learner. These data were used in conjunction with the data in Table 7 to develop summary 
profiles of the seven learners. These summaries are presented In Table 8. Accuracy was 
used to divide the students into successful (MR and GL), moderately successful (HS and 
DW), and less successful learners (II, EH, and LH). The profiles capture the relationship 
between this classification and level of expertise, length of solution (based on the total 
number of cognitive actions) and degree of monitoring (based on the total number of 
monitoring actions). 

Insert Table 8 about hero 

The profiles help to illustrate the complex interactions of text and learner that 
were obsen/ed. The two most successful learners (MR and GL) differed in the length of 
solution, with GL engaging in twice as many metacognitive and cognitive processing 
activities as MR. Whereas GL was high knowledge, he did not have MR's level of expertise 
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and may therefore have relied more on the presented information than did MR. The two 
moderately successful leamers, HS, the content expert, and DW, a novice geology student, 
replicated the differences between GL and MR. Thus, lacking expertise in the domain, 
learners who did well tended to extensively process the written material In the context of 
the question and to keep careful track of their performanc3. 

In contrast, the three less-successful learners had In common a tendency to answer 
quickly. I.e. to engage in relatively short solutions. Comparison of EH and II revealed 
identical solution success rates but showed that metacognitve processing was twice as likely 
for the lower knowledge learner. Finally, accuracy was lowest for the low knowledge, native 
English speaker, LH; she engaged in short solutions and virtually no metacognitive behavior. 

The characterizations portrayed in these summary profiles suggest that successful 
learners may be sensitive to the need for strategies that compensate for low knowledge In a 
domain. Among the present learners, the primary means of compensation was relatively 
lengthy text processing accompanied by monitoring and evaluation of the processes and 
products of solution. Nonnatlve English speakers who are sensitive to this compensatory 
mechanism, and attempt to use It, may be heavily dependent on text negotiating strategies 
and on the necessary English language skills. In contrast, nonnatlve English speakers with 
high or expert levels of knowledge in an area would be less dependent on the language of the 
text; in fact such expertise might assist students less proficient in English in leaming 
English processing skills in their content courses. 

Summary and Implications 

There were several important results of the analyses of strategies for question 
answering. First, we found a great deal of similarity among the solution strategies of the 
native and nonnatlve English speakers, consistent with previous examinations of 
comprehension strategies of native and second language readers (Block, 1986a,b). 
However, one important difference was a tendency for the nonnatlve English speakers to rely 
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more on the text than the native English speakers. Second, the more successful learners, 
I.e., those who tended to get the correct answers, engaged in cognitive monitoring, especially 
of their answers or candidate answers. Unsuccessful solutions were associated with largely 
unmonltored memory searches and this was a particular obstacle for one of the "novice" 
native speakers. Finally, the task requirements, captured In the analysis of the relation 
between question type and solution strategy, affected the characteristics of the solution 
strategies. The simplest strategies occurred for the verbatim questions, which had the 
highest rate of successful solution. The hardest question type was the one requiring 
vocabulary conversion and/or paraphrases to match the text and the question. 

The protocol analyses also revealed that text searches and comprehension 
monitoring were guided, in part, by rhetorical devices that signal various organizational and 
semantic information to the reader (e.g., Lorch, In press; Durdn, Goldman, & Smith, 
1989), In particular, there was one section of the Oceanography text that used the 
sequential markerc first and second Both native English and ESL speakers were observed 
to use the term Second to ask themselves If they knew what was first. Other types of 
rhetorical devices mark the semantic relations among the different elements of information, 
e.g., however, in addition, as a result. Understanding the semantic relations signalled by 
such rhetorical devices aids the process of constructing a coherent, internal representation 
of the text. 

We undertook a series of experimental studies that systematically pursued issues 
of comprehension and use of rhetorical devices to guide text processing and information 
acquisition. Specifically, the work had two foci, each of which was concerned with potential 
differences between native and nonnatlve English speakers. The first focus was on the 
effects of a specific rhetorical device, sequence markers, on reading behavior and memcry 
for the information read (Goldman, 1988a, b, c). The second focus was on students' 
understanding of the usage conditions for four types of connectors: additives, causals, 
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adversatives, and sequentials. We wished to determine if the four types of connectors were 
differentially difficult for readers to understand and whether there were strategic 
differences in how students performed on a rational cloze task where the deleted words were 
connectors (Goldman & Murray, 1989). 

III. STUDIES OF THE EFFECTS OF SEQUENCE MARKERS ON THE READING AND 
RECALL BEHA VIOR OF NATIVE ENGLISH AND ESL STUDENTS 

Four experiments examined the effects of sequence markers on the reading and 
retention performance of ESL and native English speaking college students. However, prior 
to conducting these studies, we detennined that the sequence marker manipulations did 
create passages that were differentially easy to understand (Refer to Experiment 1 In 
Goldman, 1988). Three studies described here (Experiments 2, 3, and 4 in Goldman, 
1989) pursued the readhg and information retention issues. Experiment 2 examined recall 
of target information by monolingual, native English speaking students. Experiment 3 
examined reading behavior and retention by ESL students and Experiment 4 looked at reading 
and retention by native English speaking students. In these three studies, the contrast was 
between retention of four major points when they were each marked by the presence of an 
enumeration term versus when they were not marked. In an additional study of reading and 
retention (Experiment 5), we examined the effect of enumerating some but not other major 
points within the same text. 

Materials 

The materials for the Experiment 1-4 were identical. Sixteen introductory texts 
containing instances of enumerated items were drawn from the following domains: cultural 
anthropology, geography, sociology, economics, psychology, oceanography, political science, 
biology, physical geography, and film studies. The texts were then modified to reflect four 
conditions that varied in their use of signalling devices. After modification, all passages 
were oetween 350 - 450 words in length, contained 19 or 20 sentences, and ranged in 
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difficulty level from grade 10 thru 12, with two at the grade 8 level, according to the Flesch 
Reading Ease Grade equivalent index. 

There were four signalling conditions for each passage. All four versions had the 
same title, began wilh the same three introductory sentences, and concluded with the same 
three sentences. A sample text is shown in Table 9. The four signalling conditions were as 
follows. 

Insert Table 9 about here 

1. Full signalling (Full): The topic sentence in the passage explicitly cued the 
number of target points to be discussed. Then the first target point was given and was 
explicitly marked by the word First This sentence was followed by two sentences that 
elaborated It. The next target point followed and was explicitly marked by the number word 
Second. It, too, was elaborated with two sentences. Similarly for target points three and 
four. 

2. Topic sentence enumeration cue (Number): The topic sentence explicitly cued 
the number of target points to be discussed. However, within the body of the text, the 
Individual points were not explicitly marked, i.e., the words first, second, third and rourth 
did not occur In the text. Otherwise the text in this condition was identical to the Full 
version. 

3. Vague enumeration cue (Vague): This condition was identical to condition 2 
except that in the toplu sentence a vaoue quantity term was used instead of the specific 
number, e.g., several appeared in place of the word Ism in the topic sentence. 

4. No signalling (None): In the fourth condition, the topic sentence contained no 
reference to the number of points to discussed. The body of the text was identical to that 
appearing in the Number and Vague conditions. 
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Experiment 1 involved the collection of normative, rating data on the passages. The 
results of this study, which is fully described In Goldman (1988b), Indicated that idea 
development was easier to follow In the Full signaling passages, and that they were more 
comprehensible than passages In the other three conditions. On the other hand, ratings of 
ease of understanding the vocabulary did not differ, consistent with our hypotheses about the 
locus of sequence marker effects. 

Experiment 2: Effects of Sequence Markers on Recall by Native English 

Speaking Students 

The purpose of Experiment 2 was to examine the effects of signalling on 
Information ratentloh. Retention was measured by recall of the target points in the passages 
in response to a probe-sentence cue. It was expected that recall of target Information would 
be greatest for texts presented in the Full signalling condition, that the Number topic 
sentence condition would be superior to the Vague; and, that each of these would be better 
than None, in which there was no signalling of the target points. 

Method 

Thirty-two Introductory Psychology students participated in the experiment as 
part of the course requirement. All were native English speakers. 
Materials. Design, and Pronftriiire 

The 16 passages used in Experiment 1 wore divided into two sets using a matched 
pairs procedure and the data from Experiment 1 . Four versions of each passage were 
assigned to four different booklets using a Latin Square counterbalancing procedure to 
Insure that each booklet contained two instances of each condition. Across booklets and 
across subjects each passage occurred in ail four conditions an equal number of times and 
each subject received two different passages in each condition. The design was a mixed 
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design with set the between -subjects factor (2 levels) and signalling condition the wlthln- 
subjects factor (4 levels). 

In addition to the eight experimental passages, each booklet contained one practice 
passage that contained no signalling. Subjects were told to read the passages carefully 
because they would later be asked to recall the Information. The task was illustrated with 
the practice passage. The topic sentences from the None condition were used to cue recall of 
the target points, e.g., In the passage about tha Ideal State, what were tha hasic issues that 
concerned Plato? Recall was done after every four experimental passages. Subjects were 
run in small groups in sessions lasting 1.5 to 2 hours. Details of scoring can be found in 
Goldman (1988b). 

Results and Discussion 
The results of ANOVAs on target point recall in each conrimon, treating subjects as 
a random factor, are reported here; analyses treating passage as a random factor are 
reported In Goldman (1908b). Treatment condition was the within-subjects variable and 
passage set was the between-subjects variable. Effects or interactions that are not reported 
were nonsignificant. 

For target point recall, there was a main effect of signalling condition, £ (3, 90) - 
10.47, Q. < .001, Msgrror » -567. The means for the signalling condition effect are shown 
in Table 10. Four post-hoc contrasts on the signalling effect were done and the Bonferroni 
procedure was used to control for the Type I error rate (for four contrasts alpha level = 
.0125 per contrast). Target point recall was significantly better in the Full signalling 
condition (M « 2.5 out of 4) than in the other three (M. « 1.65), £ (1, 90) » 30.5, q, < 
.001. Recall In the Full condition was greater than in the Numbt-' (M » 1.73) condition, £ 
(1, 90) - 8.36, ji < .01. 

Insert Table 10 about here 




Despiie the nonsignificant pairwise contrasts comparing Number, Vague, and None 
conditions, the data in Table 10 indicate that these means were ordered in the predicted 
direction. That is, recall was somewhat higher in the Number condition than In the Vague 
and None conditions and the Vague condition was slightly better than the None condition. 

Thus, as predicted, target point retention was better if passages contained explicit 
enumeration of the target information than if these signalling devices were absent. The 
pattern of data across the other three signalling conditions indicated much weaker effects of 
a signalling device that was present only in the topic sentence. Experiment 2 thus 
demonstrated the basic effect of signalling on retention. In Experiments 3 and 4 we pursued 
effects of signalling on reading behavior. 
Experiment 3: Effects of Sequence Markers on Reading and Recall Behavior 

of ESL Students 

The purpose of this study was to examine the effects of sequence markers on the 
recall and the reading behavior of ESL students. The experiment was identical In design to 
Experiment 2 except that students read the material one sentence at a time on a 
microcomputer screen. They were permitted to "flip" backwards and forwards through the 
text. We recorded this sequence of behaviors. Also, the time spent reading (viewing) each 
of the segments comprising the complete text was recorded. 

r^Gthod 

Sixteen ESL students were paid $5 per hour to participate in the study. They were 
recruited from ESL courses at a university in Southern California. The courses from which 
the students were recruited are at the intermediate level of English language skills and 
students represented a range of language backgrounds. For additional information on the 
characteristics of the students, refer to Goldman (1988b). 
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Materials. Design, and ProRadure 

The passages were the same as those used in Experiment 2, Thay were again divided 
into two sets, each set containing 8 passages, Thus, there was one between-subjects factor 
(passage set) and signalling condition (4 levels) was the within-subjects factor. Dependent 
measures were number of target points recalled, reading rate per word and processing time 
per word. The rate and processing measures were computed over the entire passage as well 
as over the specific target point sentences. 

Students viewed each of 8 passages, one sentence at a time, on a Macintosh Plus 
microcomputer screen. Specially developed software, Readit! . recorded the amount of time 
spent on each sentence and tracked the subjects fonward and backward movement through the 
sentences In the text (Saul, Pohl, & Goldman, 1988). As In Experiment 2, recall followed 
four texts and was cued by the topic sentence from the None condition. Students were tested 
individually In sessions lasting approximately 2 hours. 

Results and Discussion 

Recall 

ANOVA on the mean number of target points recalled (max » 4) Indicated a 
significant effect of condition, £ (3, 42) » 8.14, u < .001 , Msgrror - -447. The means for 
the condition effect are shown in the middle panel of Table 10. Post-hoc contrasts indicated 
that Full signalling led to greater recall than the other three conditions, £ (1 , 42) - 
13.87, u < .01. Furthermore, Number lead to greater recall than Vague, £ (1, 42) » 6.32, 
fi < .0125. The Full and Number conditions did not differ. 



ANOVA on the mean rate per word measure indicated that there were no significant 
differences in rate across the four conditions, £ < 1. Subjects read at a rate of 
approxinnately 600 milliseconds per word and all means were within 20 milliseconds of one 
another. The data are displayed in Figure 1 , For the mean processing time measure, also 
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shown in Figure 1, there was again no effect of condition, £ (3, 45) « 1.24. Subjects spent 
an average of almost one second per word. A second set of analyses specifically compared the 
processing time per word in the target point sentences when they were signalled and when 
they were not. In the Full condition, the mean process time per word was 1 .25 and in the 
None condition it was 1.06 seconds; this was a nonsignificant difference £ (1, 15) « 3.05, 
Q - .1. Despite the lack of statistically significant differences on the processing time 
measure, the magnitudes of the means are in the predicted direction and are consistent with 
the pattern of recall effects: Students spent the longost on passages with Full signalling and 
spent the shortest amount of time on the passages with Vague or No signalling. 

Insert Figure 1 about here 

Thus, predictions regarding the effects of explicitness of marking on recall of the 
target points were confirmed for the ESL students. When the specific number of points was 
cued (Full and Number conditions) recall was better than when vague or no information was 
given regarding the number of points to be developed in the text. Predictions regarding 
reading behavior were only weakly supported in that the reading rate data indicated no 
differences among the four conditions. The processing time data suggested a tendency for 
students to spend more time on the passages v/ith explicit signalling, and specifically on the 
target points. 

Experiment 4: Effects of Sequence Markers on Reading and Recall Behavior 

of Native English Speaking Students 

This study was an exact replication of Experiment 3 except that monolingual, 
native English speaking students served as subjects. 
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Method 

Sixteen native Englisfi-speaking students enrolled in an introductory Psychology 
course participated in the study. Instead of monetary payment, these students received 
credit toward their course requirements. 
Materials. Design, and Pron<?f^i|rft 

The passages, design, procedures and scoring were identical to Experiment 3. 

Results and Discussion 

Recall 

ANOVA on the mean number of target points recalled (max « 4) indicated a 
significant effect of condition, £ (3, 42) - 6.38, & < .01 , Msgrror - -358. The condition 
means are shown in the third panel of Table 10. Post-hoc contrasts indicated that Full 
signalling (M « 3.13) lead to greater recall than the other three conditions, £ (1, 42) = 
17.31, ft < .001. The other three conditions did not differ from one another (M - 2.41), £s 
<1. Recall in the Full condition was better than in the Number condition, £ (1, 42) = 7.06, 
ft < .0125. Thus, as predicted, recall was better the more explicit the signalling. 
Reading Time 

ANOVA on the mean rate per word indicated a significant effect of condition, £ (3, 
45) » 3.18, ft < .05, Msgrror » 8- Post-hoc contrasts indicated that in the Full condition, 
significantly more time was spent on each word read (Mn « 468 milliseconds) than in the 
other three conditions (M » 397 milliseconds), £ (1, 45) « 7.63, ft < .01. The Number, 
Vague and None conditions did not differ from one another. These means are shown in Figure 
2. Similarly, ANOVA on process time per word indicated a significant effect of condition, £ 
(3, 45) - 3.58^ c- .02, Mserror » 16. Contrasts on these means indicated that in the Full 
condition significantly more time was spent on each word (M » 662 milliseconds) than in 
the other three conditions (M - 575 milliseconds), £ (1, 45) - 5.63, ft < .025. Process 
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time per word in the target sentences was longer when they were signalled (M = .834) than 
when they were not (M - .634 in the None condition), £ (1, 16) « 5.23, ji « .04. The 
reading rate and processing time data indicate trends in the predicted direction: the more 
explicit the signalling, the slower the subjects' rates of reading and the more time they 
spent on the passage . 

Insert Figure 2 about here 

The performance patterns indicate that the primary locus of the effect '^f number 
signals is on attentional allocation during reading for the monolingual English speal<ers: in 
the Full condition the passage was processed more than in the Number condition and there 
was greater recall In the Full condition. That the Number and Vague conditions did not differ 
significantly suggests that when the signalling merely provides a criterion for recall 
completion, retention by native English speakers is not substantially enhanced. 
Experiment 5: Linguistic Proficiency and the Effects of Sequence Markers 

on Strategic Reading Beiiavior and Recall 
This experiment pursued two issues suggested by our earlier results. The first issue 
concerned the role of linguistic proficiency in the effects of the markers on recall. 
Essentially, we were interested in whether there were differences related to linguistic 
proficiency In the ability to use sequential markers to aid in retrieval of the signalled 
information. Accordingly, we sampled two levels of linguistic proficiency among no.inative 
speakers of English. The second issue concerned the more direct examination of the role of 
the sequence rarkers in comprehension monitoring. Within the same text, some sentences 
were marked with sequence terms while others were not. The specific predictions were that 
backtracking in the text would be related to disjunctures in the enumeration of four target 
points in the text. For example, in one condition, the first and third points in a four point 
text were preceded by the words Firsi or Third, respectively. When the student reads 
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"Third " if they were monitoring tlieir understanding, tliey sliould have bacl<tracl<ed to 

try and find the second point. This backtracl<ing should occur if readers use the enumeration 
signals to aid their construction of an internal representation of the text and if they have not 
already coded a second point. That is, readers with sufficient skill may recognize the second 
point even without an explicit marker, and thus absence of the explicit marker would not 
affect their reading behavior. 

Method 

S ubjects 

There were 32 monolingual native English speakers and a total of 32 ESL speakers 
who participated in this study. These students participated as part of a requirement in their 
Introductory Psychology course. The ESL speakers consisted of two proficiency levels, with 
16 students at each group. ESL 2 consisted of students who were taking the two most 
advanced courses in the English language ESL sequence. ESL 3 consisted of 1 6 students who 
were unrolled m the two Icwer level English language courses in the ESL sequence. A variety 
of native languages were represented by the students In each of the ESL classes. Asian 
languages accounted for 88% of the ESL 3 and 38% of the ESL2 students; European languages 
accounted for the remainderof the students. The majority of students in all of the language 
groups were freshmen and sophomores 
Materials. Design and Procedure 

The materials were the passages used h Experiments 1 - 4, modified to implement 
the manipulation of the location of the sequence markers. There were again four conditions, 
but they differed from those used in the previous experiments. Specific to this study were 
the fcllowing conditions: 

1. In the Number-Only condition, the topic sentence gave the specific number of 
points but none of the target points were signalled in the text. This condition was 
comparable to the Number condition of Experiments 1 - 4. 
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2. In the Number - 1 & 3 condition, the topic sentence gave the specific number 
of polntG and only the first target point and the third target point were signalled by the word 
First or Third, respectively. 

3. In the Vague - 1 & 3 condition, the topic sentence used a vague numerical term 
(e.g., several) to Indicate how many target points there were and only the first target point 
and the third target point were signalled by the word First or Third, respectively. 

4. In the Vague - 2 & 4 condition, the topic sentence was the same as in the Vague 
1 & 3 condition but only the second and fourth target points were signalled by the word 
Second or Fourth, respectively. 

Details of the procedure were similar to that described for Experiments 3 and 4. 
with subjects reading two passages in each condition. In addition to the variables described 
for those experiments, there were th-ee levels of linguistic proficiency, native monolingual 
English, ESL2, and ESL3. Dependent measures were number of target points recalled and 
processing time for the entire passage and for the specific target points. 

Results and Discussion 
Of primary interest was whether target points were recalled better when they 
were signalled as compared to when they were not. Performance in the Number - Only 
condition provided "baseline" data on recall of the four points when none were signalled. 
Table 11 provides the mean recall by each language group of each target point in three 
conditions: Unsignalled when no points were enumerated {Number-only condition). 
Signalled (enumerated points from the other three conditons), and Unsignalled (points that 
had not been enumerated in the other three conditions). The recall data from Ihe Number- 
only condition were submitted to a multivariate, reoeated measures Anova in which language 
group (Native English, ESL2 and ESL3) was the between subjects factor and target point (1, 
2, 3 or 4) was the within-subjects factor, consistent with predictions, the results of the 
Anova indicated no differences in recall among the four points, E (3, 59) < *. Language 
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proficiency had a significant effect on performance in this condition, £ (2, 61) « 3.83, a < 
,03, MSerror - 1.012. The ESL2 group (j^ - 1.1) recalled significantly more than the 
ESL3 group (M - -61), but the native English students recalled the same amount (M » .91) 
as each of the ESL groups, Tukey-Kramer procedure critical difference « .60. Note that this 
group of native English speakers generally performed more poorly (M « 46% of the target 
points) than their counterparts in Experiments 4 (M in Number condition » 64%) but at 
about the same level as the students in Experiment 2 (M « 43%). The ESL2 group 
performed at a roughly comparable level (55%) in the present case and in Experiment 3 
(61%). 

Insert Table 1 1 about here 

For comparisons of signalling effects within a passage, we used the recall data from 
the Number -1 & 3, Vague -1 & 3 and Vague -2 & 4 conditions, in the passages in these 
conditions, half the target points were enumerated and half were not. For each subject we 
constructed a mean for each target point when signalled and a mean for each when 
unsignalled by averaging appropriately over the three conditions. For example, recall for 
target point 1 -signalled was the average of point 1 recall in two conditions: Numbor - 1 & 3 
and Vague -1 & 3; target point 1 -unsignalled was derived from recall in one condition: - 
Number - 2 & 4. Similarly, the data on target point 2-signalled were based on performance 
in the Number - 2 & 4 condition; target point 2-unsignalled was based on average recall in 
two conditions: Number -1 & 3 and Vague - 1 & 3. These derived data, shown in Table 1 1 , 
were submitted to a multivariate repeated measures Anova in which language group was the 
between-subjects and there were two within-subjects factors: signalled/unsignalled and 
target point (1, 2, 3, or 4). The only significant effect was language group, £ (2, 61) » 
5.46, a - .01. Mean recall by native English (M - .93) and by the ESL2 (M « .94) groups 
did not differ but was higher than recall by the ESL3 group (M « .60), although the 
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differences failed to exceed the Tukey-Kramer critical value of .6, 12 < .05. It is interesting 
to note that the patterns reflected In the means In Table 1 1 suggest that marking was more 
important for the second and fourth points than for the first and third In the two more 
English-proficient groups of students. The processing time data indicated a similar pattern 
of suggestive but nonsignificant results. 

Given these results, we did not pursue analyses of the reading behavior. Although 
the manipulation of signalling Instantiated In this study failed to show the predicted effects, 
we believe that with longer texts and/or more complicated Information, wo might well 
observe the predicted marking effect within passages. 

General Discussion of Sequence Marker Studies 2-4 . 

The monolingual, native English speakers In Experiments 2 and 4 showed the same 
pattern of effects for the sequence marker variable: Whether passages were presented in 
booklet or computer form, the Full signalling condition produced the highest recall of the 
targeted Information. The ESL students' performance was also consistent with our 
predictions: recalling the most from passages presented In the Full signalling condition. 
However, patterns of significant differences among the four conditions appeared to differ 
somewhat for the native English and ESL students and imply differences in the primary locus 
of the effects of these signals. For the native English speaking students in both experiments, 
recall was significantly better In the Full condition than in the Number condition but the 
Number and Vague conditions did not differ. At the same time, the more explicit the 
signalling, the longer the native English speakers spent reading the passages. This pattern 
suggests that for the native English speakers, the prirhary locus of the signalling effect was 
on the amount of time they spent reading the passages; there was a weaker, secondary effect 
due to the provision of a criterion for recall completion. In contrast, for the ESL students 
retention in the Full and Number conditions did not differ significantly but the Number 
condition led to better recall than the Vague condition. The pattern for the ESL students 
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suggests that knowing how many target points to lool< for and subsequentiy recail was the 
primary locus of the signalling effect and effects on attentional allocation during reading 
were secondary. 

However, the lack of a significant effect on attentional allocation in the reading of 
the ESL students and the presence of an effect in the native English speakers should not mask 
the fact that the ESL students spent almost a full second on each word whereas the native 
English speakers spent 33% to 50% less time on each word. The ESL students may exert 
very high levels of attention to any passage they read, regardless of how explicit the 
signalling. Because they are reading at what are essentially "ceiling- levels of attentional 
allocation, there may be no "room- for an effect of signals on reading time. For native 
English speakers reading these passages probably does not require maximum levels of 
attention and they can respond differentially to different levels of signalling. In fact native 
English speakers may become dependent on conventional graphic and linguistic signailing 
devices to the point where they pay attention only to information that Is marked as 
important. 

Although the retention perfo: ce of both native English and ESL students was 
better the more explicit the signailing In the passage, the procise nature of the effect of 
signals appears to be different for the two groups. For the ESL students, the signals 
apparently had little effect on their reading but may have provided them with a criterion- 
cue. I.e., the number of points, that was useful during jncoding and/or retrieval. There are 
two aspects of the data in support of this claim: (1) the amount of time spent processing 
enumerated points was not different from that spent on unenumerated points, and (2) when 
the topic sentence gave the specific number of points (Full and Number conditions) recail 
was equal and exceeded recall when there was only a vague or no Indication of the number of 
points. In contrast, the signals primarily affected the reading behavior of the native English 
speakers. Having a critorion-cue in the absence of specific signals to the points was not 
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effective with these students. Unless they road the target points longer, recall was equal. 
The data in support of this claim are (1) native English speal<ers spent more time 
processing target points when the specific points were enumerated compared to when they 
were not enumerated, and (2) recall when the topic sentence gave the number of points but 
the points themselves were not enumerated was lower than when they were enumerated. 
Thus, for the native English speakers the signals appwared to foster selective attentional 
focus during reading; the criterion-cue vrlue of the signals was weak. On the other hand, 
the ESL students may have been relying on the criterion-cue value as a check on whether 
they had gotten from the text what they should have. They read the text nonselectively until 
they had four points from the passage. Regardless of the different foci of the signalling 
effects, the present research indicates that the presence of enumeration signals improves 
the retention of the marked Information for both native English speakers and speakers of 
English as a Second Language. 

Analyses of the Reading Strategies of Native English and ESL Speakers 
The procedures used in Experiments 3 & 4, yielded a trace of the sequence in which 
the passage sentences had been read by each student. We used these traces to examine the 
strategies students were using when they read the sequence marker passages. We were 
particularly interested in whether the strategies used by native English speakers were 
different from those used by the ESL speakers. From the traces created by the Readltf 
program, we characterized the reading strategies that students were using. Initially, we 
undertook these analyses expecting that there would be differences in strategy related to the 
manipulations of the sequence marking in the passages. This turned out not to be the case for 
the most part. Furthermore, native and nonnative English speakers were far more similar 
in their strategy profiles than they were different. However, several interesting findings 
emerged regarding the identification of reading strategies, variability within individuals and 
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between-group comparisons. (For a more extended discussions of tuis work, refer to 
Goldman, (1988c)). 

From the traces of stude its' reading behavior, we identified three types of global 
strategies and a variety of local strategies. The global strategy Once Through, shown In 
Figure 3, is a straightforward, sequential reading of each sentence in the text from start to 
finish. The other two global strategies each involve going back and rereading various 
sentences In the text. The Review Strategy, shown in Figure 4, involves a straight through 
read of the passage to the end (as In Once Through), followed by rereading of some or all of 
the passage. The third one, the Regress Strategy, (Figure 5) involves backtracking prior to 
reaching the end of the passage on the first pass through it. After rereading the end of the 
passage, there might be backtracking and rereading of additional parts of the passage. 

Insert Figures 3, 4, 5 about here 

The eight passages of each of the subjects in Experiments 3 and 4 were classified as 
one of the three types of global stragegies and proportions for each subject of each strategy 
type were computed. With respect to the issue of whether different strategies were 
associated with the different signalling conditions, there was no effect of condition. That is, 
of the total number of passages that were read with a Once Through strategy, they were 
spread evenly across the conditions; similarly for Review and Regress. This was true for 
both ESL and native English readers. Figure 6 shows the mean proportion of passages in 
each strategy type for each language group. Both ESL and Monolingual English students most 
frequently used the Regress strategy- - backtracking during initial reading of the passage. 
The ESL stud'^nts were somewhat more likely to use the Review sirategy; the native English 
the Once Through strategy. However, all but one student used more than one type of 
approach and about a third of the students used all three strategies. 
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Insert Figure 6 about here 

Thus, the global strategy data Indicate that all the students varied tfie way in which 
they read the texts. But this variation was not related to the structural manipulation that 
we had done: the Regress Strategy was just as likely for Full signalling passages as for No 
signalling passages. W« pursued the reading strategies further for two reasons: (1) to 
determine If signals affected reading behavior at a more micro, sentence-to-sentence level, 
and (2) to determine what sorts of structural and semantic/conceptual characteristics of 
the passage were guiding the reading behavior. 

To examine the more micro level, we identified ten local or backtracking 
strategies, as shown in Figure 7. These local strategies are ways of describing the reader's 
movement through the text on a sentence by sentence level and capture patterns of forward 
and backward movement through the text. The ten represent two dimensions of reading 
behavtor: whether people road or skimmed and how much of the text they covered in the 
process of backtracking. In this figure, heavier lines indicate reading, lighter lines indicate 
skimming. Closed circles represent reading a single sentence. 

Insert Figure 7 about here 

The first four strategies (A, B, C, & D) reflect movement through the text in one 
direction over a relatively long string of sentences. Strategies A and B involve sequential 
movement through the passage over at least 15 consecutive sentences; A involves reading 
those sentences, B skimming. Strategy A implies a general desire to reread most of the text, 
either from beginning to end or vice versa. Strategy B was typically used to return to the 
beginning of the passage prior to going through it again from the beginning. Strategies C and 
D reflect the use of both reading and skimming over segments at least 12 sentences long. In 
C, readers skim one or more sentences, read a sentence, continue skimming one or more 
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sentences, then read a sentence. In D, readers skim one or more sentences, but read several 
sentences, before they resume skimming. 

The other six strategies reflect backtracking in which readers reverse the 
direction of their progress through the text, using a combination of reading and skimming 
and covering relatively short runs of sequentially ordered sentences (2 to 11). Of 
particular interest Is Strategy E, which- we refer to as a local rereading strategy: the reader 
is reading along but goes back and rereads the prior sentence before continuing to read. Use 
of E suggests that readers need clarification of a just-prior sentence before going on in the 
passage. 

The frequency distributions of the local strategies were similar for the two 
language groups and signalling conditions. However, the distributions varied with the type 
of global strategy. Figure 8 shows the distribution for the Review Strategy. The four single 
direction strategies - A, B, C, & D - accounted for 80% of the local strategies, indicating 
that backtracking behavior? that occurred after reading the passage through once tended to 
involve returning to the beginning or near beginning of the passage (data for B) and going 
through the passage again either by reading each sentence (data for A), or by selecting 
certain groups of sentences to read (data for D). Thus, even after getting through the 
passage once, additional inspection of the text did not tend to involve local rereading 
strategies that interrupted movement in a single direction (none greater than 10%), 

Insert Figures 8 & 9 about here 

In contrast, the local strategies used in reading Regress Strategy passages reflected 
a relatively high incidence of backing-up-to-go-forward, as shown in Figure 9. Prior to 
reaching the end of the passage, the local backtracking strategy - E - accounted for 55% of 
the backtracking tor these passages for both ESL and native English speakers. On these 
passages, students appeared to be trying to resolve comprehension difficulties and coherence 
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relations prior to taking in additional new information from the passage. The strategy A 
frequency indicates that about 15% of the time the local backtracking occurred either at the 
beginning (first 4 or 5 sentences) or end of the text (last 4 or 5 sentences) and then 
students read straight through the remainder of the text. After the first time through the 
passage, the distribution of local strategies was quite variable but generally indicated a 
greater reliance on skimming the text, with selected rereading of sentences and segments of 
the text. 

Although local strategy use was not specifically related to signalling condition, it 
was clear that the traces of the students rer ding behavior were an excellent vehicle for 
examining relationships among a variety of text characteristics and reading behaviors 
common to several local strategies. In particular, we were interested in which sentences 
initiated and ended backtracks. Initiations and conclusions of backtracks are reflected in 
direction changes in the reading traces. Locations of direction changes were characterized 
with respect to how they reflected structural and semantic characteristics of text. 
Structural aspects of text are things like beginning, ending, and explicit rhetorical 
markers, such as our enumeration terms. We were particularly interested in whether 

enumeration of the target points was an important cue to backtracking. Semantic 
asp^ois refer to various processes that must occur to achieve coherence, e.g., resolution of 
anaphor, construction of bridging inferences, etc. 

The types of sentences that initiated direction changes were similar for the ESL and 
Native English speakers but differed across the global strategies. For the Review strategies, 
about 70% of the direction changes were related to a structural characteristic of the text - 
reaching the end or beginning of the passage (Figure 10). The topic sentence and tho iarget 
points accounted for less than 20% of the direction changes. However, and for the ESL 
student only, the tendency for target points to initiate direction changes was strongest if 
they were explicitly marked (I^ jII condition). 
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Insert Figures 10 & 11 about here 

For the Regress Strategy passages (Figure 11), direction changes that occurred 
before getting to the end of the passage were governed more by semantic than structural 
properties: about 60% of the target points initiating direction changes were in the 
unmarked conditions (Number, Vague and None). This trend was true for both the ESL and 
the native English speakers. For the Regress strategy passages, after reaching the end of the 
passage the first time, direction changes appeared to be related to more structural 
properties of the texts: about 50% were initiated by reaching the beginning or end of the 
passage. The structural cue of marking was important for the native English speakers: 
explicitly marked target points accounted for 60% of the direction changes cued by the 
target points. This tendency was not present in the ESL students' data. 

Thus, the direction changes were cued by structural and semantic properties of the 
texts, depending on the particular global strategy that the students were using. We also 
found that both structural and semantic aspects of the text were related to the sentences that 
students strategically selected to reread. These data are shown in Figure 12 for passages 
read with the Regress Strategy. About 65% of the sentences strategically read before getting 
to the end of the passage the first time weie in the introduction or were target points. Once 
again, explicit marking of the points was not necessary for them to be singled out for 
selective rereading. Furthermore, the points elaborating the targets were strategically 
reread almost as much as the target point. Because these elaborations were not structurally 
marked and are at the lowest level in a content structure, they imply strategic rereading 
guided by semantic concerns. 

Insert Figure 12 about here 
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The strategy analyses indicate thai ESL and native English speakers have the same pool of 
available strategies. Furthermore, they appear to select a range of strategies and the 
variables that govern their selection are similar. However, additional research is needed to 
explore more fully the processing and strategic mechanisms that underlie the observed 
strategies. Currently, Goldman and Saul (in progress) are pursuing additional work on 
reading strategies that will enlighten our understanding of the constraints and decision rules 
that govern students' selection of one strategy over another as related to syntactic and 
semantic features of the text. 

IV. UNDERSTANDING THE USAGE OF C ONNECTORS BY NATIVE ENGLISH AND ESL 

SPEAKER? 

Sequence markers are but one type of connector used in writing a cohesive text or 
In generating a coherent representation of a text. Other types of connectors include 
additive terms that indicate elaboration or exemplification of already mentioned content, 
causal terms that indicate cause-effect or antecedent-consequent relations, and 
adversative terms that indicate that the new information contrasts with or contradicts 
previously presented information. In previous research with ESL speakers, Maclean and 
d'Anglejan (1986) examined connectors as one of a larger class of cohesive devices. They 
presented a rational cloze task to native English and ESL speakers: the deleted words were 
various types of cohesive terms, including a variety of connector terms. h/lacLean and 
d'Anglejan found that ESL speakers did not differ from native English speakers in their use 
of information from other sentences (across sentence information) to determine 
appropriate completions. 

In three experiments, we used the rational cloze procedure in conjunction with a 
multiple choice response task to examine differences in difficulty among four types of 
connectors, namely additives, causals, adversatives and sequentials (Goldman & Murray, 
1989). The studies compared the performance of native English speakers and several 
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groups of ESL students v;ho varied in level of language proficiency in English. In the first 
experiment, we compared cloze task and verbal justification performance of monolingual, 
native English speakers and ESL university students. Success in the university setting is In 
part determined by the ability to learn from text and we were interested in the levels of 
proficiency with connectors that might be characteristic of this population. Based on the 
findings from Experiment 1 on differential difficulty among the connector types, the 
purpose of Experiment 2 was to determine if monolingual native English speakers' 
confidence in their answers also varied with connector type. Finally, in Experiment 3, a 
group of comm jnity college ESL students who were at tower levels of English language 
proficiency than the university students completed the rational ctoze task, provided 
confidence ratings, and gave verbal explanations for their responses. The design of the three 
experiments can be understood by reviewing previous relevant research. 

Language Proficiency and Connector Understanding 
Passages vary in the degree to which the inter- and intrasentential relations 
contained therein are made explicit. In the absence of explicit signals to togical 
relationships we rely on a "conversational postulate" for text: we intt5r cohesion based on the 
default assumption that successive sentences are rel^'.ted, making use of other cohesion 
devices such as referential and lexical overlap (Halliday & Hasan, 1976; Kintsch & van 
Dijk, 1978; MacLean & D'Anglejan,1986). Connectors, or conjunctives (Halliday & 
Hasan, 1976), are a type of coheston device that make explicit the logical relations among 
sentences. Common connectors are and, but, however, because. Connectors are of limited 
utility unless the reader understands how connectors function in text md the logical 
relationship each specifies. This aspect of language proficiency is particularly important to 
students who acquire English as a second language (ESL) and are attempting to learn from 
English language texts. Anecdotal reports from teachers of ESL students frequently indicate 
that learning the appropriate use of connectors is an extremely difficult aspect of the 
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English language. To a lesser extent, this can also be said of native English speal<ers. For 
example, in a study of good and poor readers. Bridge and Winograd (1982) reported that 
both groups found it more difficult to provide explanations for the appropriateness of 
cohesive ties that they established with connectors than those that they established with 
referential and lexical cohesion devices. However, good readers were better able to justify 
their responses than were poor readers. 

Although Bridge and Winograd found that connectors were difficult to explain, good 
readers were better at explaining them than poor readers. There is also evidence that 
benefits derived fr -m the explicit inclusion of connectors in texts vary with language 
proficiency. For example, Geva and Ryan (1985) found that positive effects of including 
connectors occurred for skilled and less sl<illed readers when the conjunctions wers included 
and highlighted in the text but only for the skilled readers when the conjunctions were 
present in the text but not highlighted. Furthermore, Geva and Ryan (1985) found that the 
omission of connectors negatively affected less-skilled readers but had no effect on skilled 
readers. 

A major limitation of prior research on connectors is the unsystematic and limited 
sampling of annectors. It seems reasonable that some interclausal and intersentential 
relations may oe easier to infer than others and the benefits of including explicit connectors 
would therefore be differential. In addition, grammar textbooks for ESL students typically 
devote large sections to sentence and clause connectors, providing elaborate taxonomies of 
connectors. For example, Celce-Murcia and Freeman (1983) have provided a functional 
classification scheme for logical connectors that differentiates among four major types of 
connectors: additive, causal, adversative, and sequential. The first three typscaily relate to 
Interclausal or intersentential relations. Additive relations are those that signal some form 
of elaboration of previous content (e. g., In addition, That is). Cause-effect or antecedent 
consequent relations are signalled by causal connectors (e.g., As a result, Due to). 
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Adversative connectors signal contrastive elaborations (e.g., However, On the contrary). 
Sequential connectors are used to signal a more diverse set of logical relations and refer to 
the larger discourse contjxt more often than do the other connectors. Sequential connectors 
are used to enumerate lists of items (e.g., First, Second, Finally), to mark a sentence that 
previews the remainder of the text (e.g.. In stiort) or to indicate temporal sequence of 
events (e.g.. Subsequently, Latei). 

Accordingly, in this phase of the project, three experiments were conducted to 
investigate two primary questions: (1) whether the four types of logical connectors were 
differentially difficult for readers to understand, and (2) whether ESL speal^ers' 
understanding of connector usage was different from that of native English speakers. With 
respect to the first question, our expectations were that sequeniial connectors would be 
more difficult than the other three types - f connectors for all students because sequentials 
generally require reference to the more global discourse context whereas the other three do 
not. Among the other three connector types, all indicate &ome form of continued elaboration 
of previous information. However, they differ with regard to what they signal is the 
connection of new to previous information, with causal and adversative connectors signalling 
more specific logical relations than additives. The latter indicate only that the next piece of 
information adds "in some way" to the previous information. In contrast, as noted 
p.cjviousiy, causal and adversative connectors, respectively, signal causal or logical 
contingencv or contrast between old and new information in the clauses joined by the 
connectors. Predictions about the relative difficulty among these three connectors were not 
made. 

We expected that ESL speakers would less frequently supply the appropriate 
connector when compared to native English speakers. However, two issues were of greater 
interest than the overall level of performance. If the four connector types did differ in 
difficulty, was there similarity in the patterns of Ihu native English and the ESL speakers? 
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Second, we were interested in the strategien used to arrive at connector choices and whether 
these were similar for native Engiish and ESL readers, especiaiiy where differential ieveis 
of English language proficiency were present. We pursued the strategy question by focusing 
on whether response justifications reflected differences amont the connectors that are cited 
by text linguistists, discourse analysts, and ESL teachers. We collected retrospective verbal 
reports from students explaining their choices. We expected that the native English 
speakers would be sensitive to the functional, semantic differences among the connectors and 
this would be reflected in their response justifications. However, we also expected ESL 
students to be aware of usage rules because they are specifically taught. 

In all three experiments, subjects completec x rational cloze task that had 
multiple-choice response options (as opposed to free completion). Confidence ratings were 
collected in Experiments 2 and 3. Students provided explanations of their responses in 
Experiments 1 and 3. Ail native English speakers were drawn from the same population. 
The ESL speakers came from two populations that differed in English language proficiency 
levels. 

Experiment 1: Effects of Connector Types on the Cloze Completions and 
Response Justifications of Native English and ESL Speakers 

The purpose of Experiment 1 was to compare the performance of native Engiish and 
ESL university-ievei college students on a rational cloze task that required them to 
distinguish among four logical connectors in choosing the most appropriate intersententiai 
connector. The rational cloze procedure was applied to passages characteristic of 
introductory-level textbooks used in university settings. Our general prediction was that 
ESL students would make fewer appropriate choices than native Engiish speakers. We 
expected the differential difficulty of the four types of connectors to be similar in each 
group of students. Finally, based on previous research we expected that ESL and native 
English v^peakers would be similarly aware of prescriptive rules on the usage of the various 
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connector types but that the ESL students would be less accurate In idintlfying where in text 
the various rules applied. 

Method 

Participants were 16 monolingual, native English speal<ers and 20 ESL speakers, 
recruited from undergraduate classes at a university in Southern California. The native 
English speakers were enrolled in Introductory Psychology and participated as part of the 
course requirements. The ESL students were enrolled in English-language classes 
specifically designated for nonnative speakers of English. Successful completion of these 
classes Is a prerequisite to satisfying the university's General Education English 
requirement. ESL students who participated in this study were volunteers recruited from 
classes at three levels, but the majority (15 students) came from the class (English 1) one 
level below the university-required English class. ESL students were paid $5 per hour for 
their participation. 

Several demographic characteristics were assessed using a questionnaire filled oui 
at the beginning of the experimental session. A summary of information most pertinent to 
the present study is provided in Table 12. The native English speakers were largely college 
freshmen and sophomores and all spoke English as the native language. The mean verbal 
Scholastic Aptitude Test (SAT) Score of 526.8 indicates that these students were 
representative of the entering freshmen classes at this university over the past several 
years. The ESL students were largely f eshmen and sophomores. The native language of 
70% of the ESL students was an Asian language (Chinese, Vietnamese, Japanese or Korean). 
The verbal SAT scores for the ESL speakers are typical for this population. Performance on 
the Test of Englit i as a Foreign Language (TOEFL) is the language proficiency indicator 
typically used in making admissions dQcisions. The moan ncore of 564.7 is representative of 
students from non-English speaking countries who are admitted to the university. 
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Insert Table 12 about ,iere 



Design and Materials 

The design was a mixed factorial with one between-subjects factor (native English 
or ESL language group) and one within-subjects factor (connector type). There were four 
levels of connector type: additive, causal, adversative, and sequential. Each connector type 
was the correct response twice in each of 4 passages for a total of 8 observations per subject 
on each connector. In addition to examining the number correct, we asked subjects to 
explain their choices for two of the passages. Passages were presented in four different 
orders, according to a Latin-square design. Within each language group, each presentation 
order was used an equal number of times. 

Passages . Four passages from college-level, introductory science and social 
science textbooks were modified to accommodate 2 instances of each of the connector types. 
The mean number of words per passage was 488 and the mean number of sentences was 
32.75. The passages were about biological characteristics of life, anthropological 
approaches to the study of culture, oceanographic explorations in the nineteenth century, 
and the societal functions of marriage. For purposes of illustrating the task to subjects, an» 
additional, shorter passage (about the movie industry) was developed. 

Subjects' task was to read each passage and to choose an appropriate word to I'll in 
each of the cloze slots (represented as blanks) that occurred throughout the passage. Each 
cloze slot occurred at the beginning of a sentence and four alternatives were provided. Each 
half of each passage had four slots in it and each slot required a different connector type as 
the correct answer. A minimum of one complete st.Uence separated successive slots. The 
alternatives wera instances of the four connector types; the correct response for each cloze 
slot was determined by the experimenters. That the experimenter-designated response was 
the best choice for the slot was verified by a panel of three independent judges, one of wl.om 
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is an ESL spacialist. The order of the alternatives was systematically varied so the correct 
choice occurred in each position (1, 2, 3, or 4) an equal number of times. The order of the 
distractors was also varied so instances of the same connector type did not always appear in 
the same position. 

Our classification of connector types and the specific instances used as response 
options were taken from the functional classification scheme proposed by Celce-Murcia and 
Freeman (1983) and based on the earlier work of Halliday and Hasan (1976). For the 
additives we used simple additive instances (e.g., In addition) and exemplification instances 
(e..g., for example); for causals, we used instances that signal cause/reason (e.g., as a 
result) and effect/result (e.g., thus). For adversatives we used instances that signal that 
two ideas are in contrast or conflict (e.g., howevei) and we used instances that signal a 
reservation or restriction in the applicability of the preceding information (e.g., despite). 
Sequential connectors featured the largest variety of subtypes and we used instances that 
signalled enumeration of points chronologically (e.g., second) or temporally (e.g., next) as 
well as those that indicated summation of information (e.g., In short). 

The passages (including the practice passage) were arranged in a single "passage" 
booklet, and the alternatives for the slots for each passage were arranged in a separate 
"response" booklet. The order of the four passages was counterbalanced such that, across 
all subjects, each prssage appeared in each position in the booklet an equal number of times. 

Procedure. Subjects were run individually in sessions lasting 1.5 to 2 hours. 
Each subject completed a backgiound questionnaire at tho start of the session. After filling 
out the questionnaire, each subject was instructed on how to coordinate the passage and 
response booklets and worked through the practice passage during which time they could ask 
questions about the task. For each blank, subjects circled the tiasl word to complete the 
sentence from among the four alternative words that were provided in the response booklet. 
Subjects were told that all punctuation associated with the altornatives had been 
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intentionally omitted, but that they could add any punctuation they felt was necessary to 
their response. 

Following completion of the four passages, the experimenter re-presented the first 
two passages (excluding the practice passage) the subject had read. Subjects were told that 
we were Interested in how they had decided on their answer and, in why they thought their 
answer was the best choice for the particular blank. For each blanl^, the students explained 
their reasons for selecting their alternative choices. This part of the session was audiotaped 
and later transcribed. 
Scoring 

Each subject received a score from 0 to 8, indicating the number of correct 
responses to the 8 cloze slots for each connector type. We determined the frequency with 
which each of the three distractor alternatives was selected incorrectly. For the incorrect 
responses we also examined the contingency relation betwean the correct connector and the 
type of distractor chosen. In the verbal reports, students tended not to described the process 
they went through in choosing the answer; rather they consistently described the type of 
relationship that held between the items of information that were being connected. To score 
these responses a set of 1 1 coding categories was developed. The categories reflect the 
previously discussed distinctions among the connectors; each coding category was 
appropriate for only one of the connector types. Interrater reliability in scoring the 
responses was better than 90%. Further details on the scoring are available in Goldman and 
Murray (1989). 

Results 

Correct Respons^p 

The number correct for each subject for each connector type was subjected to a 
multivariate, split- . ANOVA in which language group was the between-subjects factor 
and connector type the within. The means for each connector type and language group are 
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provided in Table 13. As expected, the main effect for language group was significant, £ (1, 
33) = 15.28, p_< .001, Mserror « 5.02: The native English speakers were correct more 
often (M « 6.34) than the ESL students (M » 4.88). In addition, there was a main effect 
for connector type, £ (3, 32) » 3.27, .03. Five post hoc contrasts, using the 
Bonferroni procedure (alpha level « .01), were conducted. Additives and causals v^ere 
correctly answered more often than were adversatives and sequential, £ (1 , 34) = 8.46, q 
« .006, Mserror - 6.69. The other four contrasts were not significant nor was the 
interaction between language group and connector type, £ (3, 32) = 1.58. 

Insert Table 13 about here 

Distractor Chninfis 

The correct response data indicated that the ESL speakers made more errors than 
the native English speakers - a mean of 12.2 per student (out of 32 possible) compared to 
6.62. Regarding these errors, the following issues were of interest: (a) which connectors 
made the best distractors, i.e., were most frequently chosen, and (b) whether the 
distractors chosen by the two groups were similar. To investigate these issues, we computed 
the proportion of each subject's total errors that reflected each distractor option. The 
proportion data were subjected to an arr^ine transformation and then submitted to a 
multivariate, split-plot ANOVA with language group the betwen-subjects factor and 
connector type the within. There was no main effect for language group, £ (1, 34) « 3.04, 
B « .09. However, there was a main effect of connector type. £ (3, 31) = 10.06, q_<. 001; 
Additives and causals were chosen significantly more often than adversatives and 
sequentials, £ (1, 33) = 9.48, c » .004. There was also a significant interacticii of 
language group by connector type, £ (3, 31) = 4.6^, ft = .001. Table 14 gives the mean 
proportions for each distractor and language group. Examination of the means suggested that 
the interaction was due to differences in the magnitude of the differences among distractcis. 
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Accordingly, three difference scores were computed for each subject in each group and the 
differences between the distractors were tested for significance, using the Dunn>Bonferroni 
procedure to control the Type 1 error rate (alpha « .017). Causals were chosen 
significantly more than adversatives by each group but the difference was larger in the 
native English speakers, 1 (32) « 2.83. The ESL speakers chose the ad^yersatives and 
sequentials equally often but the native English speakers chose the sequentials more often 
than the adversatives, 1 (32) » 2.46. Although the means suggest that the native English 
speakers chose causals more frequently than additives while the ESL speakers chose 
additives more than causals, the difference between the groups failed to reach conventional 
levels of significance, 1 (32) = 1.88. 

Insert Tables 14 & 15 here 

An additional analysis pursued potential dependency relations between the errors 
and the correct connectors. The presence of a strong dependency, reflected by a dominant 
distractor for a particular connector type, would suggest some type of systematic 
misunderstanding of the meaning and usage of that connector. The proportions shown in 
Table 15 reflect the number of times each of the distracior options was chosen relative to 
the total number of errors on each connector. These dependency matrices indicate a 
relatively strong relationship between additive cloze slots and causal distractors for both 
the native English and the ESL speakers (probability of choosing a causal if an error Is made 
on an additive cloze slot « .76 and .60, respectively). For each of th. other connector types, 
Incorrect responses tended to be split between two distractor options. 

Both native English and ESL speakers were most frequently correct when additive 
or causal connectors were required by the text; they also choose these two types of 
connectors most often when they responded incorrectly, and there was some evidence of a 
dependency relationship between tho additive cloze slots and the causal diytractor. The 
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higher correct and distractor choice rates for the additives and causals suggest the 
possibiiity that subjects were generaiiy biased toward choosing additive and causai terms. 
We examined this by determining the rates of choosing each of the four connector types as 
responses, regardless of whether the response was correct or Incorrect. Then we 
determined the percentage of choices that were indeed correct. These data are presented in 
Table 16. 

insert Table 16 about here 

According to the design of the passages, each connector should have been selected 
25% of the time. The data indicate that the native English speakers and the ESL speakers 
selected the causals and the additives more frequently and the adversatives and sequentials 
less frequently than a "completely correct" response profile. However, the probability of 
being correct, given the choice of a particular distractor, was higher for the adversatives 
and the sequentials than for the additives and causals. It appears that both native English and 
ESL students have a more stringent set of constraints governing the appropriateness of 
adversative and sequential terms than those that govern additive and causals. 

JustificatiQji£j.p.r.„QJialfiaa 

Justifications for Choices that were Corrent. Table 17 indicates the distribution of 
correct-choice justifications over the coding categories. The proportions were computed 
separately for each connector type, using the total number of justifications for the specific 
connector as the base. There are several interesting aspects of these data. First, the 
distribution of justifications for the ESL students is highly similar to that of the native 
English cpeakers: for each connector type, the proportional distributions over the coding 
categories for the connector reflect the same pattern. When there was a dominant response, 
as in the case of the causalu, the adversatives, and the sequentials, the proportions for the 
ESL and the native English speakers were almost identical. Second, for the adversatives and 
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the sequentials over 95% of the justifications were accounted for by connector-appropriate 
categories. The proportion of connector-appropriate justifications was somewhat iower for 
the additive and causai connectors, aithough more than 75% were in connector-appropriate 
categories. 

Insert Tabies 17 & 18 about here 

Justifications for Choices that were Innorrftnt. The distribution of the 
justifications for incorrect choices are shown in Table 18. The confusions among connectors 
that were reflected in the contingency analysis of the errors (Table 15) were generally 
reflected in the justification data. When students were justifying incorrect responses that 
had been made to additive slots, the majority of the justifications were in categories 
appropriate to causal connectors. This is consistent with the data showing that causal 
distractors were the most frequently chosen for additive slots. The most frequent 
justifications for incorrect choices for causal slots were in categories appropriate for 
additive connectors, with the remaining responses distributed over the causal-appropriate, 
adversative-appropriate and miscellaneous categories. The additive and causal distributions 
for native English and ESL speakers were highly similar. 

There were some differences between the language groups in the distributions of 
justifications for incorrect responses made to adversative and sequential slots. For 
adversative slots, the native English speakers used categories appropriate to the causal 56% 
of the time and categories appropriate to the additive 13% of the time. In contrast, *ho ESL 
students used causal- and additive-appropriate categories equally often (34% each). The 
native English speakers' justifications of incorrect choices for sequential slots tended to be 
in causal-appropriate categories most frequently and additive-appropriate or suquential- 
appropriate categories equally often (24% each). On the other hand for incorrect 



ERIC 



64 



61 

sequentials, the E3L students used additive-appropriate categories most frequently, causal- 
appropriate less often, and rarely used sequential-appropriate jui^tifications. 

The distractor dependency analysis and the justification data considered separately 
suggest that errors may be due to a "fuzzy" understanding of certain of the connectors. To 
more precisely pursue the source(s) of difficulty on items that students answered 
incorrectly we examined the relation between the justification and the connector selected. 
Three relations are informative with respect to source(s) of difficulty and these are 
Illustrated in Figure 13. The first relation - incorrect alternative but a justification 
appropriate to the alternative chosen - indicates that the error is cue to diffitjulty 
processing the information in the text: the student has selected the connector that matches 
the relation extracted from the text but it is not the logical relation actually called for by 
the text. For example, students' understanding of the text may have made them think that a 
causal connector was needed in an additive slot; students chose the causal and explained their 
choice using a causal-appropriate justification. Thus, they supplied the right cornectcr for 
the wrong relation. This was the dominant pattern for both the native English speakers 
(63% of 56 opportunities) and the ESL speakers (65% of 122 opportunities). 

Insert Figure 13 about here 

The second type of relation between the connector selected and the justification - 
incorrect alternative but with a justification appropriate to the cloze slot - implies 
difficulty with the meaning of the connector terms: the student has extracted the logical 
relation called for by the text but does not choose the connector that conveys that relation. 
For example, the student explained the (incorrect) choice of a causal saying that what was 
needed in the text was an additive connector. This pattern did not account for many of tiie 
errors: 16% for the native English speakers and 6% for the ESL The third relation 
reflects problems in procGSsing the text ml in matching the connector terms with the 
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inferred relation. In this case, the student selected an Incorrect connector and provided a 
justification that was appropriate to neither the selected connector nor the cloze slot in the 
text. This pattern accounted for a moderate amount of the native English speakers' responses 
(21%) and for a somewhat higher percentage of the ESL students' responses (30%). 

Thus, difficulty extracting the appropriate logical relation is implicated as an 
important source of student errors: inaccurate inferences about the appropriate logical 
relation between information contained in iiuccessive sentences in the text accounted for the 
vast majority (95%) of the ESL students' errors and 84% of the native English students' 
errors. The data also indicate that a simple lack of knowledge" of the functions and 
meanings of various connector-type instances was not a primary reason for incorrect 
responses. 

Discussion 

The results indicate that native English speakers correctly completed more of the 
cloze slots than did the ESL students. For both groups, the pattern of difficulty among the 
connector types was similar: cloze slots requiring additive and causal connectors were more 
likely to be filled in correctly than were cloze slots requiring adversafve or sequential 
slots. This pattern of differential difficulty partially confirmed our expectations. Although 
we had expected sequentials to be the most difficult, we had not expected the adversatives to 
be as difficult as the sequentials. Justifications for correct responses were similar for the 
two language groups and reflected the distinctions among connectors that we had postulated. 

When incorrect responses were made, differences, as well as similarities, emerged 
between the language v,. . !<5 and among the connectors. !n general, when native English 
speakers were incorreci tney completed the cloze slots with causal connectors whereas the 
general tendency for the ESL students was to choose additive connectors. When these choices 
were examined contingent on the correct response for the cloze slots, we found that when 
additives were the correct choice, both language groups had primarily chosen causals. Both 
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groups showed similar choices when they incorrectly responded to causal and adversative 
cloze slots. However, the two groups differed on their incorrect choices for sequential cloze 
slots, with the ESL students primarily choosing additives while the native English students 
selected both causals and additives. 

In both the ESL and native English speakers, there were tendencies toward overuse 
of the additive and causal connector terms. And although students were correct most 
frequently on additive and causal cloze slots, when they did choose adversative and sequential 
instances, there was a high probability that these choices were correct. Taken together, 
these findings suggested the possibility that students' perceived confidence about the 
appropriate logical relationship and choice might be less when they chose additives and 
causals as compared to when they chose adversatives and sequentials. We pursued this issue 
in Experiment 2 by administering a confidence rating scale along with the forced-choice 
cloze task. 

Experiment 2: Effects of Connector Types on Cloze Completions and 
Confidence Ratings of Native English Speakers 

Experiment 2 tested the hypothesis that students would be more confident of 
adversative and sequential cloze slot compleuons than of additive and causal completions. 
This study also served as a replication of the basic findings of Experiment 1 with a new 
sample of native English speakers. The methods used were similur to those of Experiment 1, 
except that confidDnce ratings were made and no justification data were collected. 

f^ethod 

Thirty-two native English speaking undergraduates enrolled in an Introductory 
Psychology class at a university in Southern California participated in this study for class 
credit. The demographic data reported on the background questionnaire indicated profiles 
similar to those of the native English speakers in Experiment 1. 
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Design. Materials and Prot^ftdurft 

The materials and design were identical to those used in Experiment 1 with the 
addition of a confidence scale that was printed on every page of the response bool<let. This 
scale was printed In number line form and ranged from 1 (very low confidence) to 7 (very 
high confidence). Beneath each number was printed a word reflecting the degree of 
confidence represented by that number. The procedure was similar to Experiment 1 except 
that all subjects were run together in a single group session and provided confidence ratings 
rather than justifications. The design was a within-subjects design in which connector type 
was a four-level factor. 

Results 

Correct Responses. 

As in Experiment 1 , the number of correct responses for each type of connector 
was computed for each subject. The results of a one-way, multivariate, repeated measures 
ANOVA Indicated a significant effect for connector type, £ (3, 29) » 8.86, q < .001 . Four 
post hoc contrasts of the means shown in Table 19 were computed, using the Bonferroni 
procedure (alpha » .0125). As in Experiment 1, additives and causals were correctly 
answered more frequently than v/ere adversatives and sequentials, £ (1, 31) = 13.4, ja < 
.001, Mserror = 6.8. Causals were correct more frequently than additives, £ (1, 31) = 
8.02, c = .008, Mserror « 2.43. The difference between adversatives and sequentials was 
not significant, £ (1, 31) = 1.37. Also shown in Table 19 are students' confidence ratings 
for those items that were correct. A one-way ANOVA on the confidence ratings indicated a 
significant effect of connector type, £ (3, 29) » 8.8, a < .001. Post hoc comparison of the 
additive and causal ratings with the adversative and sequential ratings confirmed our 
prediction: Confidence ratings were higher for correctly chosen adversatives and 
sequentials than for additives and causals, £ (1, 31) = 23.75, b < .001, Mserror = .771. 
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Insert Tablo 19 

Distractor Cholcas. 

As in Experiment 1, we computed the distribution of each subject's total errors 
over the distractor alternatives. These proportions were transformed using the arcsine 
transformation and submitted to a one-way, multivariate, ANOVA. There was a significant 
effect of connector type, E (3, 29) - 14.65, ft < .001. The mean proportions and 
transformed values are provided in Table 20. Four post hoc comparisons (alpha = .0125) 
Indicated that additives and causals were chosen as distractors more frequently than 
advsrsatives and sequential, £ (1, 31) » 44.85, < .001, Mserror = .772. Causal 
distractors were chosen more frequently than additives, £ (1, 31) 7.57, ft = .01, 
Mserror » -605. Adversative and sequential distractors were chosen equally often. 
Confidence in these incorrect choices was not significantly different across the four 
connector types, £ (3, 60; = 2.16, ft « .1. 

Insert Tables 20, 21 & 22 about here 

The results of analyzing the dependency between the correct connector types for the 
cloze slots and the type of distractor selected when errors were made are given in Table 21 . 
The pattern replicates the pattern observed in Experiment 1. The causal connector was the 
dominant distractor for additive cloze slots and the distributions for the adversative and 
sequential cloze slots were virtually identical \o those given in Table 15. The one difference 
between the results of the two experiments is that more adversative distractors were chosen 
in causal slots by the students in the second study. The selection rate data, shown in Table 
22, indicated a bias tovjard causals in that 32% of the choices weru causals; however, only 
66% of those choices were correct. Adversatives and sequentials wero chosen least 
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frequently but had the higher percent correct rates. Copfidence ratings for correct 
responses were highest for the iess frequently chosen connector types. 

Discussion 

The results of Experiment 2 replicated the findings of Experiment 1 and extended 
them by showing that students were more confident of cloze completions for adversative and 
sequential connectors. Adversative and sequential connectors appear to have more restricted 
and perhaps dearer usage conditions than do additives and causals. Causals and additive 
terms, such as and and so may be more frequent in everyday language and their usage may 
not always literally connote a highly restricted meaning compared to adversatives and 
sequentials; as a result, some of their more specific meanings may become "diluted" in text 
more so than the meanings of adversatives and sequentials. Students may choose additives 
and causals because they are more familiar in everyday speech, not because they are certain 
about their appropriateness In the contexts of written text. Hence, even when their choices 
are correct, students' confidence In additive and causal completions is weaker than their 
confidence in less frequently chosen adversative and sequential connectors. 

These data provide support for the interpretation that although adversative logical 
relations and sequential temporal relations are less often constructed by students, when they 
are constructed, students are confident of the accuracy of their Inferences. There is 
significantly more doubt when causals and additives are selected for use in a text. 
Experiment 3 pursued these issues with ESL students at generally lower levels of English 
language competency than the individuals who participated in Experiment 1 . Because of 
this, we also used a new sample of texts appropriate to this population and a somewhat 
different set of connector terms. 
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Experiment 3: Effects of Connector Ty^3 on Cloze Completions, Confidence 
Ratings, and Response Justifications of ESL Students 

The purpose of Experiment 3 was to examine whether ESl students confidence in 
their cloze compietions varied with the type of connector. In addition, we were interested In 
testing the replicability of the ESL results from Experiment 1. Due to constraints on the 
ESL population in our locale, the sample for Experiment 3 was drawn from a local two- 
year, community college rather than from the university. Because this population is 
generally at lower levels of English language proficiency than the university popuicition, we 
developed a new set of pass«ges based on texts at Jie level of the students' proficiency in 
English. Thus, Experiment 3 was a replication/extension of the basic findings of 
Experiments 1 and 2 to both a new subject population and different materials. Data were 
collected on the cloze completion, the confidence rating and the response justification tasks. 

Method 

M2j££t& 

Participants were 35 ESL students enrolled ir. the advanced-level ESL English 
course at a local community college. Nineteen of the students were enrolled in a course that 
met during the day and 16 attended a night class. Ail students in the class particip-^vted in the 
paper and pencil part of the iMiik and it was conducted during class time. Following the group 
session, 8 students from the day class and each of the night class students were seen 
individually for purposes of explaining their response c hoices on the first two passages that 
they had read. 

The ESL stbdents in Experiment r were more heterogeneous than the ESL students 
who participated In Experiment 1 . As is evident from the data in Ta Jie 23, the subjects in 
the present study were older and reflected a greater age range. They had taken fewer related 
courses and the iuvei of English language skills (M » 5.3 grade level equivalent) was lower 
than for the ESL students in Experiment 1, who were reading at levels sufficient to enter the 
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university, in addition, aii except two of the subjects for Experiment 3 had started 
schooiing in Engiish at age 12 and 50% had actuaiiy started at 18 years of age or iater. 

insert Tabie 23 about here 

Design and Matfirlalfi 

The design was a mixed factoriai in which ciass (day or evening) was a between- 
subjects factor and connector type {four leveis) was the within-subjects factor. Dependent 
measures were derived from the cloze compietion responses, the confidence rating tasl< and 
the justification data. Due to the generaiiy iower ievei of Engiish sl<ili in this popuiation of 
ESL students as compared to the university students of Experiment 1 , new passages were 
developed. The same constraints on the occurrence of bianl<s and distribution of response 
alternatives described for Experiment 1 and 2 were followed in constructing the passages 
U3'^d in experiment 3. 

PassaaeSi The four passages were modifie 1 versions of texts drawn from textbooks 
that had been used in the past by the ESL program at t e community college but not by the 
particular students who participated in the study. We selected passages that dealt with fields 
in the social and natural sciences. Specifically, the following topics were discussed: life 
sty'es during the Paleolithic Age, emotions that cause laughter, the natural resources in 
Siberia, and ihe mystery of the Bermuda triangle. After modification to accommodate the 
connectors, the mean number of words per passage was 518 and the mean number of 
sente -cey per passage was 32.25. For three of the passages, the Flesch grade level 
equivalent was 9-10 and for 1 it was 7-8. 

A passage bool<let and accompanying response bool<let were created for each subject. 
The passages were presented in four different orders to counterbalance position effects for 
the experimuntal passages. The practice passage was always the first one. As in Experiment 
2, each page of the response bool<let contained the four alternative choices for a particular 
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cloze slot as well as a confidence rating scale. The same constraints described for 
Experiments 1 and 2 on the ordering of the response alternatives were followed in this 
experiment. 

Eigcedure 

The subjects enrolled in the day class completed the passage and response booklets 
over a series of 4 class sessions, each session lasting for 55 minutes. The subjects enrolled 
in the evening class completed their booklets over a series of 2 class sessions each 
approximately 2 hours in duration. The procedure involving the booklets was identical to 
the one used in Experiment 2. After completing the booklets, a sample of 8 students from 
the day class were seen in individual sessions and asked to explain their response choices for 
the first two passages. Subjects were systematically selected so that a discussion of each 
passage occurred an equal number of times. All subjects enrolled in the evening class were 
interviewed. 

Results 

■Correct Responses 

The mean probability of correctly selecting each type of connector was computed 
for each student. A multivariate, split-plot ANOVA revealed no main effect (£ (1 , 33) = 
2.93, n > .05) nor interaction (E < 1) involving the class variable. However, consistent 
with Experiments 1 and 2, there was a main effect of connector type, £ (3,31) = 13.76, q.<. 
001 . Five post hoc comparisons were computed using the means shown in Table 24 and the 
Bonferroni procedure was used to control the Type 1 error 'aie (alpha = .01). Additives 
and causals were correctly chosen more frequently than adversatives and sequentials, £ (1, 
33) » 36.27, £ < .001, Mserror » .075. Additives and causals were not significantly 
different, £ (1, 33) « 3.47; nor were adversatives and sequentials, £ (1, 33) = 3.45. This 
pattern replicates the result obtained in Experiments 1 and 2: Responses for additive and 
causal slots were more often correct than were those for adversative and sequential slots. 
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Insert Table 24 about here 

In contrast to the findings of Experiment 2, the ANOVA of the confidence rating data 
failed to reveal any significant differences due to connector type, £ (3,93) = 2.19, n = .09, 
Mserror " '24. Thus, the ESL students were no more confident of their adversative or 
sequential choices than they were of their additive and causal choices. Furthermore, their 
confidence ratings for their correct answers were about 1 scale value below the rating? 
the native English speakers' confidence In their correct answers. 

The proportion of each subject's total errors associated with each distractor 
alternative was transfomed using the arcsine transformation and submitted to a 
multivariate, split-plot ANOVA in which class and connector type were factors. As in the 
analysis of the correct responses, there was no effect of class, £ (1, 33) = 3.51, ft = .07. 
There was, however, a significant effect of connector type, £ (3, 31) - 20.77, n <.001. 
The means are shown In Table 25. Five post hoc comparisons, using the BonferronI 
procedure (alpha level - .01), were conducted. Additive and causal distractors were 
selected more frequently than adversative and sequential distractors, £ (1 , 33) = 51 .77, ft 
< .001 , Mserror = -381 . Causal distractors were selected significantly more often than 
additives, £ (I, 33) « 13.23, ft < .001, Ms ers jr « .24. The probabilities of choosing 
adversative and sequential distractors were nut significantly different from one another; nor 
were the probabilities of choosing additive and adversative distractors. The distractor 
choice data replicate the findings for the native English speakers in Experiments 1 and 2 in 
that causals were the most frequently chosen distractor. For the ESL students in Experiment 
1 there was a tendency to choose the additives and causals most often; thus there was a minor 
difference between the two ESL samples. 
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Insert Table 25 about here 

ANOVA on the confidence ratings for the incorrect cloze completions revealed a 
significant effect of connector type, £ (3, 29) = 3.96, ^ < .01 . Post hoc contrasts indicaied 
that confidence was higher for adversative distractors than for sequential distractors, £(1, 
31) « 11.24, a » .002. Examination of the patterns of the means shown in Tables 24 and 
25, the confidence ratings foi the correct responses, suggested that there was a potentially 
Interesting interaction between connector type and ratings for correct compared to 
incorrect choices. The ANOVA on these data indicated higher confidence when the correct 
alternative was chosen (M » 4.93) than when a distractor was chosen (M = 4.57), £ (1, 
28) = 14.29, a = .01. There was also a significant connector type by correctness 
Inteiaction, £ (3, 26) = 3.64, ft = .02. The interaction was pursued with three post hoc 
contrasts (alpha level « .017). The significant contrasts indicated (1) that the difference 
between ratings of correct and incorrect responses was larger for sequential connectors 
than It was for the other three connector types, £ (1, 28) « 8.63, ft = .007, Mserror = 
8.57; and (2) that correct selections of sequentials received higher confidence ratings than 
when sequential distractors were chosen, £ (1, 28) = 17.33, ft < .001, Mserror = .9/1. 

The results of Experiments 1 and 2 suggested that both native English and ESL 
students often selected causal connectors when ariditives were most appropriate. In the 
present sample of ESL students, the dependency elation analysis indicated a much weaker 
relationship. As the data in Table 26 indicate, 50% of the distractor selections for additives 
were causals but adversatives were selected in almost 40% the cases. In addition, additivy 
and adversative distractors were the most frequent distractois selected when errors were 
made on causal slots; finally, causals were the most frequently selected distractor for 
adversative and sequential slots. It appears that these ESL students who were at lower levels 
of English language proficiency had a strong tendency to overattribute causality. 
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Insert Table 26 & 27 about here 

That there was a general bias toward selecting causal connectors is supponad by 
the analysis of the connectors selected by the students. The data are provided in Table 27. 
Rrst, 34% of the choices were causals, reflecting the tendency to choose causal 
alternatives. However, the likelihood that :hese choices would be correct was relatively 
low, 42%. Sequentials were least often chosen but had the highest likelihood of being 
correct. Additives wer3 given the highest confidence ratings, in contrast to the pattern 
manifest by the native English speakers In Experinfient 2. Thus, although the adversative 
and sequential alternatives were selected .jast often, they did not manifest the pattern of 
proportion correct and confidence ratings that prevailed in Experiment 2. Nor did the 
present sample of ESL students manifest the pattern shown by the ESL students in 
Experiment 1, wherein adversative and sequential alternatives were selected least 
frequently but were most frequently correct. It is possible that these ESL students who are 
at less sophisticated levels of Enlgish language training do not make use of the greater degree 
of constraint governing the use of adversative and sequential connectors. The verbal 
justiiications were pursued to enlighten the criteria that these students were in fact using. 
Justifications of R esponses 

Justifications for Corrfint Responses . Table 28 provides the proportion of 
responses In each of the justification categories for each of the connectors. Several trends 
are important. First, the major.iy of justifications for each of the connectors were in 
categories consistent with definitions and taxonomies of logical connectors, i.e. 60% or more 
of the responses were in connector-appropriate ctitegories. Furthermore, the dominant 
responses within each category were consistent with the distribution obtained from the 
native English and ESL four-year college students in Experiment 1. Justifications for 
additive slots 'vere divi ler, between example and elaboration. The dominant justification for 
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causal slots was the existence of a cause-effect relation; uie dominant justification for 
adversative slots Indicated that comparison, contrast, or unexpected Information was 
present. For the sequential slots, justifications Involving new or next points or temporal 
relations were given. There was some difference between the pattern on the sequentlals In 
this study and the patterns In Experiment 1; however. In designing the texts for Experiment 
• 3 we purposely tried to use connectors that Indicated temporal relations or "next" points. 
Thus, differences In the justification data for Experiments 1 and 3 are undoubtedly due to 
the specific passages and connectors we Included In the materials. 

There was one difference between Experiments 1 and 3 that was probably not due to 
the specific passages: Approximately 37% o. the justifications for correct responses were 
in the Miscellaneous category and the majority of these were choice by exclusion, i.e., "The 
others didn't fit." The higher frequency of the exclusionary category suggests that the 
community college students may be operating with a greater degree of Implicit or tacit 
knowledge about these connectors than are the university students. The former may be 
trying the alternatives to see which "sounds" best, relying on implicit knowledge In the 
process. This knowledge may be unavailable to them for verbalization. 

Justifications for the incorrect responses, As with the justifications for correct 
responses, about 30% of the justifications of incorrect responses were in the miscellaneous 
category, with no difference between connectors in this trend. Consistent with the results of 
Experiment 1, the proportion of justifications for choices that were Incorrect generally 
reflected the dependency relations that were reflected In Table 26. For example, 36% of 
the 44 justifications for Incorrect causal-slot responses were additive-appropriate reasons 
and 18% were adversative-appropriate. For adversative slots, 27% were additive- 
appropriate and 27% were causal-appropriate categories. For Incorrectly complete 
sequential slots, causal -appropriate explanations were provided 41% of the time. The 
additive slots where the only ones wore subjects showed more than a slight tendency to use 
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connector-appropriate connectors: 23% of the justifications for incorrect responses were 
additive-appropriate. 

The justification data were used in conjunction with the completion data to examine 
the source(s) of difficulty for these students. Of the three relations discussed in the context 
of Experiment 1, we found two of them to be equally likely in the data of the ESL community 
college students. For 48% of the justifications for incorrect responses, students used a 
justification appropriate to the distractor they had selected (branch 1 in Figure 13). This 
pattern implies difficulty understanding the relation called for by the text. An additional 
45% of the justifications for incorrect responses used a justification that was 
inappropriate given She slot in the text ami did not fit the distractor selected (branch 3 in 
Figure 13). This pattern implies difficulties both in processing the relations in the text and 
in understanding the meanings of specific connector terms. Justifications that were 
inappropriate to the choice but appropriate to the slot were relatively rare (6%) and 
Indicated that knowledge of the specific connector words was not the major source of 
difficulty. Thus, the major problem seemed to be in processing logical relations in context 
and recognizing when the specific types of logical connectors were needed. 

Discussion 

The patterns of results in this study were generally consistent with our 
predictions and the results of those of Experiments 1 and 2. Cloze completion responses 
were more frequently correct for additive and causal slots than for adversative and 
sequential slots and justifications for the correct responses reflected the appropriate 
connector functions. However, contrary to expectations, the ESL students' confidence 
ratings for correct choices were equivalent to one another. When they were incorrect, ESL 
students most frequently ch')se causal distractors, although their confidence in these choices 
was not terribly high. The confidence rating data for the incorrect choices was 
unanticipated: they were more confident of wrong answers when they had chosen 
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adversatives and additives than when they had chosen caus^ls and sequentials. Finally, the 
dependency relations between the correct cloze completion and the type of connector 
incorrectly selected, in conjunction with the explanations of the incorrect choices suggest 
that in this sample, the ESL students had a less precise understanding of the differences in 
meaning between frequently used causal expressions, sucn as thus, so, and hence and 
frequently confuse them with simpler additive or sequential indicators. 

General Discussion of Connector Studies 

In each of the three studies there was a consistent effect of connector type: when 
cloze slots required additive or causal corupletion terms, students were more likely to be 
correct than when adversative or sequential terms were needed. We had predicted that 
sequentials would be difficult because (1) there are a greater number of sequential subtypes 
and (2) correctly selecting one often requirus a reference to the global passage rather than 
to the local claube or sentence context. We were somewhat surprised that the adversative 
tended to be as difficult as the sequential. We suggest three plausible explanations of 
performance on adversatives. The specific instances of the adversative may have lower 
frequency of use than the instances of additives and causals. Second, performance on the 
adversative may be affected by the existence of a reader consistency bias. That is, readers 
may be operating with a default assumption that favors interpreting successive sentences as 
elaborating on the old material rather than by contradicting it or restricting its scope. 
Finally, recognition of an adversative relation may require a more complex backward search 
to prior content than the causal or the additive. 

The effect of linguistic proficiency on correct performance was conj^is;ent with our 
general expectations. ESL speakers were able to correctly complete fewer cloze slots than 
native English speakers; community college ESL speakers pei formed at lower levels than 
university ESL speakers. However, it is important that the connector type pattern was 
generally consistent across groups. 
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That there was a response bias toward the additive and causal instances v/as 
illustrated in two of the measures. First, causal and additive connectors were the nriost 
frequently chosen distractors. Furthermore, the student selection rate data showed a strong 
tendency to choose causals and additives. The selection rate analysis also showed that 
adversblive and sequentials were more likely to be correct when they were selected, and 
students had the highest degrees of confidence in these, although this effect was not as strong 
among the least English proficient. 

Analyses of the incorrect choices revealed some interesting information regarding 
Intersentential reasoning and inference making. Causal distractors were particularly likely 
when students failed to choose the correct additive alternative. Justifications for these 
errors indicated that students had Inferred a causal relation where an additive had been 
Intended. The tendency to incorrectly choose a causal was present but at attenuated levels 
for the adversative and sequential slots. The strongest trend toward choosing the causal was 
present in the data of the community college ESL students. 

The patterns associated with incorrect responding may reflect two influences. 
Readers may be using the causal in an effort to create relatively tight connections among 
units of information. Alternatively, conversational English may create a "sloppy" meaning 
for causal connectors such as so, thus, and because. That is, these terms may be used in 
situations where a relationship other than cause-effect \v> being discussed. They may serve 
as "psuedo bridges" rather than as true causals. As a result, greatest overuse of the causal 
would be expected for those students whose dominant experiences with English have btsn in 
informal, converi,ational contexts. Similar confusions in understanding the causal have been 
reported among children acquiring English as a first language (e.g., Corrigan, 1975). 
Content domain study cema>.ds more precision In the meaning of the language we use; 
connectors are no oxception. Thus, as students engage in more interaction with formal text, 
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they are forced to refine their understandings of these terms and the usage constraints that 
govern them. 

The reiationship between incorrect responses and their justifications indicated 
that errors were predominantiy due to incorrect inferences sbout the appropriate iogical 
relation and an inability to find the connector that expressed those incorrectly inferred 
relations in context. When students had inferred the appropriate relation between 
sentences, they rarely erred in choosing the correct connector. 

A major source of difficulty for students doing this task was inferring the 
appropriate relation between successive sentences. For informational texts such as these, 
designed with the explicit purpose of communicating new information to students. It seems 
particularly important to use logical connectors, especially when contrastive points are 
being made. It is also clear that in content domains where it is important for students to 
clearly distinguish between psuedo and true cause-effect relations, explicit connectors in 
the text will facilitate accurate understanding. We also want to emphasize that differences 
in performance between native English and ESL students were largely in overall levels of 
performance and were not primarily associated with differential patterns among the 
connectors. General content-domain comprehension skills, rather than specific connector 
skills, are therefore implicated as the locus of the language group differences. Once ESL 
students have grasped the basic meaning and functions of instances of specific connectors, 
further drills on isolated use of connectors are not likely to lead to improved performance 
on connecto>'s in natural te^it contexts. Rather, improving ESL proficiency at this level 
seems to require instruction that fosters understanding logical relationships between 
sentences and how connectors signal such relations when those sentences occur in 
meaningful, content-domain contexts. 
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V. SUMMARY AND CONCLUSIONS 

The research conducted 'inder the auspices of this project generally indicates that 
there are more similarities than differences in the reasoning and corrprehension processes 
of ESL and native English speakers. There were few differences found between language 
groups in the processes applied to answering questions, to reading and recalling 
Information, and to constructing cohesion between sentences of a lengthy passage. Linguistic 
characteristics of text were found to affect those processes similarly, regardless of language 
group. Our comparative investigations revealed several interesting aspects of the nature of 
learner, task, and text interactions in text processing and we turn to a discussion of these 
here. 

In the oceanography protocol study, differences between more and less successful 
learners were associated with domain-specific background knowledge rather than with 
English language proficiency. Those with greater knowledge of domain-related concepts 
performed better than those with a lesser knowledge base. We found that for both language 
groups, certain types of questions were more difficult than others. The more difficult 
questions were those that required application of information from the text to new situations 
and those that paraphrased statements in the text. The one difference we found between 
native English speakers and ESL speakers was that the latter tended to engage In more 
extensive text searches for purposes of both locating question-relevant information and for 
confirming answers. The ESL students' greater reliance on text searches emphasizes the 
importance of being able to thoroughly decode and process the linguistic form. But 
weaknesses in processing the texi do not necessarily impair question answering: ESL 
students with high levels of domain-specific knowledge may compensate for less than perfect 
proficiency in English by activating knowledge of principles and facts relevant to questions, 
based on contextual cues discfirned in the text and questions. 

Finally, the protocols also revealed an interesting "impasse" that occurred when 
questions had a high number of words that matched the text: Both ESL and native English 
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students often engaged in lengthy text searches, as if they beiieved that the answer "was 
there somewhere." Particuiariy for questions requiring application or integration the 
simple search strategy did not produce the correct answer. Having engaged in a lengthy but 
unsuccessful search for the answer, students frequently gave up Lnd did not answer the 
question. They rarely re-evaluated their approach. One implication of these impasses is 
thai application questions should be indicated as such. It would undoubtedly facilitate 
learning, especially of ESL students whose dominant solution strategy is text search, to more 
clearly differentiate between questions that are answered "in the text" and those that require 
thinking "beyond" the text. 

The oceanography protocols of both native English and ESL students indicated that 
they did not always capitalize on the linguistic devices that cue the organization of discourse. 
Our experimental studies pursued one class of such devices, connectors. As a group, 
connectors signal 'ha logical relationships among clauses and sentences and they can be 
important cues to local and global discourse organization. The experimental studies 
indicated that native English and ESL speakers' ability to recall information was aided by the 
presence of sequential connectors. However, reading behavior appeared to be governed by a 
broader range of factors than the simple presence of a sequential marker. A broad range of 
semantic and structural factors appear to operate in concert as readers proceed through 
text. It will be important in subsequent research to examine these factors more precisely. 
The methodology developed in the present context is an important tool for pursuing these 
issues. The reading strategy analyses indicated that individuals used a mixture of strategies 
for reading text. ESL and native English sneakers portrayed similar mixtures of strategies, 
although there was u tendency for the ESL students to review text materials more frequently 
than did the native English speakers. The more extensive processing of t ext in these studies 
replicates the similar finding in the oceanography study. 
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In our studies of four types of connectors, we again found that the generai 
orientation to the tasl< was simiiar for the native Engiish and ESL students. Patterns of 
correct responding as weii as of incorrect responding were consistent across groups, in aii 
the language groups, but especiaiiy in the least English proficient ESL students, there was a 
tendency to overuse causal connectors. This tendency may reflect the influence of usage 
patterns in spoken English, where speakers often use causal terms even when a cause-effect 
relation is not intended. Errors were associated most frequently with inappropriate 
inferences about the logical relation between sentences; errors were not primarily due to 
incorrect understanding of specific connector terms themselves. One implication of this 
research is that the inclusion of connectors to signal intended intersententiai relations is 
likely to facilitate appropriate understanding. The clear and explicit marking of the 
relation in this way is likely to benefit most those students for whom English langu&ge 
reasoning and comprehension are weakest. 

ESL students appear to apply reasoning and comprehension strategies that are 
very much like those used by native English speakers. A range of text and task 
characteristics have been shown to similarly affect each language group. Despite the 
similarities, the ESL speakers were less efficient than the native English speakers. The ESL 
speakers took longer to complete every one of our tasks, even though they generally 
performed at lower levels. The costs associated with more laborious and effortful 
processing on the part of ESL speakers may affect motivation and general attitudes toward 
reading and schooling as well as the efficiency with which cognitive operations are 
performed . Motivational costs may manifest themselves in a reduction in perseverance 
when faced with assignments deemed difficult even for native English speakers. 
Furthermore, because ESL speakers, even at the university level, require more time per 
assignment than native English speakers, there are potential effects on the amount of work 
that can be accomplished in a fixod period of time and on the number of courses that can be 
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taken in any given school term, l u enhance acadumic and economic performance of ESL 
speakers, important questions for subsequent research are those that inform our 
understanding of (1) cognitive, motivationai and attitudinai costs of inefficient processing 
and (2) the development of efficient i easoning and comprehension in English ESL speakers. 
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Table 1 



Question types and distribution of experimentai questions across types 



1. Verbatim relationship between the question and the text. 

2, 3A: "What are..." properties of concept 

12. 1A: "How many..." quantitative response 

12, 3A: "What fraction..." quantitative response 

12, 3C & D: "What (%)..." quantitative response 

12, 3E: "Does tre difference (between quantities)..." 

2. Paraphrase relationship between the question and the text, Including 
the necessity for vocabulary conversion and equivalence. 

2, 1A: "How do we..." 
2, 1B: "Why ..." 
2,5: "How can..." 

12, 2A: "What is..." quantitative response 

3. Verbatim "look up" plus comparison. 

12, 1B&C: "How does this compare..." quantitative comparison 
12, 2B: "How does this compare..." quantitative comparison 

12,5: "How are.. .similar?" conceot comparison 

4. Cross paragraph Integration. 

2, 38: "Explain (properties)..." concepts 

2, 6: "Explain how..." process 

12, 4: "What is the difference between..." concept comparison 

12,6: "Explain why..." concept and process. 

5. Reasoning, application, computation questions. 

2, 2: "How much..." quantitative response. Formula; not all the necessary 

information is explicitly in the text. 

12,38; "What..." quantitative response. The next value in a series must 

be determined from the series. Alternatively, the rule 
that generates the values in the series is given in the 
text and may be applied (see 12, 3E) to extend the 
series. 
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Table 2 

Subject characteristics 

Self-rating 











learning from English 


command of 


Learner 


Major 


SAT (V/M) 


Native Language 


lecture text 


science vocab 


HSa 


Geology 




English 


Extr. good Extr. gooa 


Extr. good 


MRb 


Physics 


560/650 


English 


Good Extr. good 


Qxxi 


LH 


Communications 


990-both 


English 


Extr. good Extr. good 


Good 


DWC 


Sociology 


550/590 


English 


Extr. good Extr. good 


Qxd 


II 


Bus./econ 


380/540 


Croatian 


Good Good 


Moderate 


EH 


Bus./econ. 


550/600 


Dutch 


Good Extr. good 


Moderate 


a 


History 


520/440 


Spanish 


Extr. good Good 


Moderate 



^The teaching assistant for the course and a graduate student studying Oceanography. 

^MR's background Is fieavily oriented to the hard sciences. He had already taken two other College 

level geology courses. In combination with his physics major, this background qualifies MR as a 

subject matter expert or near expert in the introductory course In Oceanograp:./ because many of 

the concepts and relationships were familiar to him from his other science courses. 

^DW had had one year of foreign study in Chile and lists Spanish, French and Russian as other 

languages that she has knowledge of . 



Table 3 

Question Answering Modei for Learning from Academic Texts 



Processing Events 



Goals 



Encode Question 



Search Memory for Answer 



Search External Source for 
Answer, e.g. Textbook 



Construct and Output Answer 



Goal: Determine tlie type of answer 
required by the question. 

Goal: Determine starting point(s) for 
searching for an answer. 

Goal: Find a candidate answer that 

exceeds criterion for response. 
(Evaluate iil<eiihood of success 
with continued search, if high, 
continue memory search; if 
iow, try another means.) 

Goal: Determine aiternate sources for 
answer. (Memory search may 
provide information that 
facilitates external search.) 

Goal: Delimit se.irch space. (Use 
question and results of any 
memory search.) 

Goal: Find relevant information. 

Goal: Process the text information 
in the context of the task 
defined by the question. 

Goal: Answer question completely. 

Goal: Match type of answer to type of 
question. 



Metacognltive Processing, 
e.g.. Confirm Answer 

Monitor quality of answer 
Self-Monitor 



93 



Table 4 

Natural language protocol for DW 3 

1 . Explain why you feel cold when you get out of the shower. 
(Reading the question) 

2. I think it's coming back to latent heat, 

3. Explain why you feel cold when you get out of the shower. 
(Rereading the question.) 

4 . Because of the evaporation process. 
(Initial Answer) 

5. But what is it that causes that? 

6. Back to the book. 

7. "Suppose we take a certain amount o. water saturc.„d air 

8. and raise its temperature by one degree Celsius. 

9. Some of the added heat would go into raising the temperature. 

10. For example, because the water is saturated, increase... 

11. We know that nearly twice as much added heat can go into evaporating water." 
(Lines 7 - 11: Reading 829-832 in section B.2) 

12. "Since the fas..." 

(Begins to read 815 in section B.2) 

13. I'm thinking in my mind at the same time as I'm reading this 

1 4. that ' could simply say because the shower was so warm, 

15. and you'd get out and the air temperature's not as warm as the shower, 

16. but I know that there's something in the book that I'm missing. 

17. Um. " 29% continents" 

18. "remainder stored polar caps..." 

1 9. "atmosphere loses..." 

20. Oh, here we go. 

21 . "The ocean loses water but gains it back through precipitation, runoff... 
"(Lines 17-21: Skimming from beginning of Chapter through section A.2 ) 

22. "Everyone is aware that the water in an open pan will slowly evaporate. 

23. The warmer and drier the air, 

24. the faster the water will disappear. 

(table continues) 
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Table 4 (continued) 

25. The amount of water the air can hoid depends on .... 
(Lines 22 - 25: Reading S1, 2 , 3 under B.1) 

26. "room temperature..." 

27. "several..." 

28. "content..." 

(Lines 26 - 28: Skimming S4 - S9 in section B.1) 

29. I guess, just to answer that simply, 

30. Well, let me check. 

31 . So "Absolute humidity is simply the percentage of air that is..." 
(Reading S10 in section B.1) 

32. "Then the absolute humidity... 

33. "!f air at 100% humidity is cooled then it becomes supersaturated 

34. and the excess moisture precipitates. 

35. The dew point is that temperature at which the air will become saturated. 

36. This is the temperature at which the air will be saturated. 

37. Notice that the dew point is... " 

(Lines 32 - 37: Sl<imming ttirougti S14 - Si 7 in section B.1, occasionally 
reading a clause of a sentence aloud.) 

38. "It takes a lot of heat to evaporate water. 

39. "We have seen that one calorie will raise the temperature of one gram of 
water by one degree Celsius. 

40. Therefom, it takes 100 calories to bring one gram of water from it's 
freezing point to boiling, 

41. but an additional 540 are required to actually evaporate the water. 

42. Thus, a pan of water on the stove readily comes to boiling, 

43. but it takes a long time to boil dry. 

44. If it Wv9ren't for this we'd have difficulty boiling an egg or making soup." 
(Lines 38 - 44: Reading SI - S6 in section B.2) 

45. Ah, hah...Wail. 

46. "Therefore, it takes 100 calories to bring one gram of water from it's 
freezing point to boiling, 

47. but an additional 540 are required to actually evaporate the water. 

48. Thus, a pan of water on the stove readily comes to boiling, 

(table continues) 

95 

ERIC 



Table 4 (continued) 

49. but takes a long time to boil dry. 

50. If it weren't for this, wa'd have difficulty boiling an egg or making soup." 
(Line 46 - 50: Rereads 83- S6) 

51 . "This is also why we perspire. 

52. The heat required to evaporate the water comes off of our skins, cooling us 
off. 

(Lines 51 - 52: Reading S7, S8 in section B.2 [end of the first paragraph in 
section B.2]) 

53. Well, I guess I'll do it in sort of a round-about way. 

54. First of all, the heat, 

55. the hot water of the shower... 

56. you get out of the shower, and the water, which is storing the heat on your 
body, is... 

57. you usually dry it off 

58. and the evaporation, um 

59. Actually, that's not true 

60. cause if it's cold outside, 

61 . the evaporation is a slower process. 

62. "The heat required to evaporate the water comes from our skin," 
(Rereading 88 in section B.2) 

63. Well, I guess I'd say evaporation and leave it at that. 

(Note: She ended up with the answer she started with, having read and 
reread various sections of the text. S17, 818 in section B.2 actually 
contains the answer.) 



3 DW's statements are in regular type face. Material that is being read from the text is in 
quotes. Italicized information is commentary. 
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Table 5 

Example of a coded protocol for DW's solution to question 12,6 



Cognitive Action 

Reads 
Identifies 
Rereads 
Recalls 

Questions 

Describes 

Reads 

Reads 

States 



Monitors/Evaluates 

Skims 

Reads 
Skims 
Describes 

States 

Reads 

Skims 

Reads 

Recognizes 

Rereads 

Reads 



Information 

question 

topic and text section 
question 

answer from memory 
self 

strategy = go to book 
text B.2 S29-S32 
text B.2S15 
answer (partially 
correct answer) 



Comment/Interpretation 

"Latent heat" 

Global answer. "Because 
of the evaporation process" 
"But what is it that causes that?" 



answer 

text from beginning of chapter 
through section A.2 
text section B.1S1- S3 
text section B.1S4-9 
next action 

need to monitor 
text section B.IS10 
text section B.1S11-17 
tex; section B.2S1- S6 
relevant information 
text B,2S3-S6 
text B.2S7-S8 



"Because the shower was so warm 
and you'd get out and the air 

temperature's not as warm as 

the shower but..." 

"I know that there's some thing in 
the book that I'm missing." 



"I guess, just to answer that 
simply..." 

"Well, let me check" 



"Aha.. .Wait" 



(table continues) 



ERIC 



9? 



Table 5 continued 



States answer that Not terribly different from 

attempts to para- initial answer, 

phrase what she's read 

Rereads text B.2S8 

Decides/States answer = original Reverts to original answer 



95 



Table 6 

Strategies for correct solutions to the five types of questions^ 

Question type 





Type 1 


Type 4 


Type 3 


Type 5 


Type 2 




(n«35) 


(n»28) 


(n=21) 


(n=1 4) 


(n = 28) 


Percent correct 


89% 


71% 


62% 


57% 


39% 


Search memory (Type A) .32 


.45 


.54 


.50 


.25 


Search text (Type B) 


.58 


.40 


.31 


.25 


.58 


Search memory 












and text (Type C) 


.10 


.15 


.15 


0 


.08 


Text search, reason 












beyond (Type D) 


0 


0 


0 


.25 


0 


No added processes 


.45 


.40 


.23 


.13 


.42 


Question analysis 


.26 


• .30 


.23 


.25 


.08 


Reasoning/Inference 


.13 


.05 


.54 


.63 


0 


Monitorina^ 


Al 




A3. 


1.13 


J3. 


Process 


.13 


.25 


.08 


.50 


.33 


Product 


.29 


.40 


.38 


.63 


.42 



^Percent correct solutions is given in parentheses tor each type of question. Probabilities in 
the body of the table are based on the frequency of occurrence of each event in correct 
solutions. 

^Underlined data in this row are the sums of the probability of process and product 
monitoring. Because both could occur in a protocol, the sums can exceed 1 .00. 
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TaDlf) 7 

Solution strategies for the seven learners^ 

Native English 



Learners 



Nonnative English 





MR 


HS 


DW 


LH 


GL 




II ■ 


Number Correct 


16 


12 


12 


8 


15 


10 


1 0 


Percent Correct 


(89%) 


\\} / /o I 


/ ft 7 0/ \ 


t A AO/ \ 


[0 0 70 ) 


/ceo/ \ 
( 0 0 70 ) 


/ceo/ 

(55% 


Search Type 
















Memory 


9(8) 


10(7) 


5(4) 


10(5) 


3(2) 


3(3) 


7(4) 


Text 


5(4) 


7(5) 


7(4) 


7(3) 


15(13) 


1 0(5) 


9(5) 


Both 


3(3) 




4(2) 


1(0) 




4(2) 


1(1) 


Compute 


1(1) 


1(0) 


1(1) 






1(0) 


1 (0) 


Additional Processing Activities 














None 


7 


5 


2 


7 


1 


5 


6 


Question analysis 


2 


2 


9 


8 


9 


6 


5 


Reasoning/inference 5 


6 


6 


5 


6 


4 


5 


Monitoring^ 


1 




lA 


I 


il 


2. 


m 


Process 


3 


8 


9 


3 


6 


8 


5 


Product 


8 


3 


9 


4 


6 


3 


6 



^Maximum « 18 questions. Entries reflect the number of questions whose solutions included 
a particular search or processing activity at least once. Numbers in parentheses Indicate 
the number of correctiv ansvyerej questions having that search type or activity. 
^These data are the number of solutions containing either process or product monitoring. 
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Table 8 

Summary profiles of the seven oceanography learners 



Successful learners 

MR (89% correct) 

Gia (83% correct) 

Moderately successful learners 
HS (67% correct) 

DW (67% correct) 

Less successful learners 
l|3 (55% correct) 

EH^ (55% correct) 

LH (44% correct) 

^Nonnative English speakers 



Knowledge 

Expert 
High 

Expert 
Low 



Medium 

Low 

Low 



Length of 
solution 

Short 

(67 actions) 
Long 
(148 actions) 

Short 
(92 actions) 

Long 
(172 actions) 

Short 
(90 actions) 

Short 
(83 actions) 

Short 
(80 actions) 



Monitoring 

Moderate 
(27 actions) 
High 

(46 actions) 

Moderate 
(31 actions) 
High 

(76 actions) 
Low 

(17 actions) 
Moderate 
(32 actions) 
Low 

(9 actions) 
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Table 9 

Sample Passage^ 
The Ideal State 

Plato was concerned with achieving systematic unity of a society. He did not believe that the 
primary role of the state was to ensure a feeling of well-being in each one of its 
participants. According to his theory, the state is a permanent organization that, as a whole, 
has definite needs and a definite inner structure, and goals that are higher than that of 
making individuals happy. In his reflections on society, Plato was concerned 
with four (several) basic issues. First, he was convinced that increasing 
individualization threatened the social order. His model of the ideal state provided for 
numerous measures that would ensure the citizens' conformity, both in their public 
behavior and in their thoughts and feelings. He provided for supervision and control of all 
aspects of individual life. Second. Plato's conception of the state addressed the problem of 
the division of labor and specialization. He argued that no human being was self-sufficient 
and so the coexistence of human beings must be based on mutual exchanges of services. The 
individual and the state would each attain wholeness and prosper only if labor was divided so 
that each man did the task for which nature had designed him. Third, Plato attached great 
importance to the optimal size of a city. He thought that the city could be either too small or 
too large, and he actually fixed the optimum population of the Greek city at 5,040. He 
strongly believed that excessive territorial expansion and increased population of the state 
posed a particular threat to its unity. Fourth, he attached great significance to the problem 
of differeniiation of wealth within the society. He saw increasing social inequalities as a 
major source of decomposition of the state because it led to the interests of small groups 
being put before the Interests of the society. He argued for the abolition of private property 
and the family, regarding both as prime sources of the problem. Plato viewed society as an 
interdependent system made up of many parts possessing their own interests. He was 
interested in social facts insofar as they had definite consequences for the integration or 
disintegration of society. Above all, he was concerned with discovering conditions that were 
conducive to social integration and equilibrium. 

^The text for the "Full" signalling condition used the bold faced information. The number 
condition used only the bold faced sentence, not the specific sequence terms, e.g.. First, 
Second, etc. The Vague condition also did not use the number terms; the topic sentence used 
the parenthesized term instead of the specific number term. In the None condition, the topic 
sentence excluded both the specific and the vague quantifier and did not use the sequence 
markers at the front of each sentence. 



102 



ERIC 



Table 10 

Mean recall of the target points (max « 4) by Native English speakers (Experiment 2), ESL 
Speakers (Experiment 3), and Native English Speakers (Experiment 4) 



Full 

Native English speakers (n = 32) 2.5 
ESL speakers (n = 2.72 

EmnnmiA 

Native English speakers (n = 16) 3.13 



Cignalling condition 

Number Vague tor-e 

1.73 1.67 1.55 

2.44 1.84 1.72 

2.57 2.29 2,38 
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Table 11 

Mean rsca!! of tho signalled and uitsionalled target sentences in Study 5^ 

Target Point 







PI 


P2 


P3 


P4 


No points marked 


Mono 


1.03 


.84 


1 .0 


.81 


(Nun-Jber-Only Condilion) 


ESL2 


1.19 


1.13 


1.0 


1 .06 






56 


. D O 


.oo 


.69 


Signalled 


Mono 


1.06 


1.13 


.92 


.91 




ESL2 


.91 


1.06 


.81 


1.19 




ESL3 


.56 


.56 


.53 


.69 


Unsignalled 


Mono 


1.0 


.92 


.91 


.73 




ESl_2 


1.13 


.57 


1.0 


.78 




ESL3 


.56 


.81 


.56 


.50 



^ The maximum score in each of the cells is 2. 
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Table 12 

Characteristics of tlie Students in Connector Experiment 1^ 



Native English ESL 

Characteristic Speal<ers Speal<ers 

Pgs Mean 18.75 20.3 

Range 18-20 18-30 

Age at which schooiing Mean - - 11.63 

in Engiish began Range - - 4-26 

Scholastic Aptitude Test (V)b Mean 526.8 338.3 

Range 450 - 600 210 - 450 

Social Science Courses Mean 2.78 3.1 

in High School or College Range 0-12 0-15 
Major Field of Study 

Social Science 31% 30% 

Natural Science, including Engineering 18% 60% 

Undecided 50% 5% 
Native Language 

English 100% o% 

Spanish 5 o/^ 

European (except Spanish) 25% 

Asian 70% 



3 All data are self-reported. We did not have access to official student records. 

Table continues 

Table 12 continued 
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^ AIM 6 Native English speakers reported Verbal SAT scores; only 12 ESL speakers 
reported Verbal SAT scores. The mean SAT score fcr freshmen at this university has 
fluctuated around 500 for the past several yea^s. An additional 7 ESL students reported 
scores on the Test of English as a Foreign Language (TOEFL) and 2 students reported both 
SAT and TOEFL scores. The mean TOEFL was 564.7, with scores ranging from 510 to 630. 
The mean TOEFL score is representative of that reflected in the nonnative English speaking 
undergraduate population at this ur'versity. 
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Table 13 

Mean Number Correct on Each Type of Connector in Connector Experiment 1^ 







Language Group 






Native English 


ESL 


Overall Connector 


Connector 


(n= 16) 


(n = 20) 


Means (n = 36) 


Additive 


6.44 


5.7 


6.03 


Causai 


6.56 


5.0 


5.69 


Adversat'Ve 


6.19 


4.35 


5.17 


Sequential 


6.19 


4.45 


5.22 


Overall Group Means 


6.34 


4.88 





^The overall means have been weighted to reflect the unequal sample sizes. The 
maximum number correct was 8. 
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Table 14 

Distractor Selections In Connector E;;periment 1^ 

Language Group 



Native English ESL 
(n= 16) (n = 20) 

Mean errors'^ 6.62 12.20 
Distractor Selected 

Additive .29 .39 

Causal .41 .31 

Adversative .05 .14 

Sequential .16 .13 



^The Anova was done on arcsine transforms of the proportion of incorrect responses for 
which each type of connector was selected. The means of the transformed measures were as 
follows. For the native English speakers: add'Mve = 1.1; causal = 1.38; adversative = .44; 
sequential = .76. For the ESL speakers: additive = 1.37; adversative = .73; causal = 1.16; 
sequential « .7. 

^The maximum number of errors was 32 (8 for each of 4 connectors). 
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Table 15 

The Proportion of Errorj> on Each Connector Type Accounted for by Each of the Distractor 
Options for Connector Experiment 1 3 



Distractor Options 

Correct Connector Additive Causal Adversative Sequential 

Native English Speakers 

Additive - - - .76 .08 .16 

Causal .43 - - - .09 .47 

Adversative .40 .43 --- .17 

Sequential .38 .41 .21 - - - 

ESL Speakers 

Additive - - - .60 .24 .15 

Causal .47 - - - .15 .38 

Adversative .49 .40 - - - .11 

Sequential .50 .29 .21 - - - 



^The proportions are based on the total number of erroro for each connector type. For the 
Native English speakers the number of errors were the following: On additive slots, 25; on 
causal, 23; on adversative, 30; on the sequential slots, 29. For the ESL speakers the 
number of errors were as follows: additive , 46; causal , 60; adversative , 72; and 
sequential , 70. 
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Table 16 

Student Selection Rate Distributions for Connector Experiment 1 



Connector 

Additives 
Causais 
Adversative 
Sequential 



Percentage of Selections^ 
Native English ESL 



27 
29 
21 
23 



33 
28 
19 
20 



Percentage of Selections 
Scored as Correct 
Native English ESL 



76 
70 
91 
83 



54 
56 
71 
70 



ajotal number of selections for the Native English students was 512 and the total for the ESL 
students was 640. 
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Table 17 

Distribution of Justifications for Correct Response", for Connector Experiment 1^ 



Connector and Justification Category^ Native English ESL 

Additives - Total number of correct responses 4 6 5 5 

Information gives example of concept .54 40 

Information elaborates prior information .24 .42 

Inappropriate justification .17 .15 

Causals - Total number of correct responses 51 4 6 

Cause-effect relationship between two sentences .71 .67 

Consistent but vaguely stated logical relationship .08 .04 

Conclusion about a cause - effect relationship that 

was developed over several sentences. .06 0 

Inappropriate justification .17 .22 

Adversatlves - Total number of correct responses 4 5 4 6 

Comparison or contrast with information In the 

prior sentence; unexpected information .84 .91 

Restriction of the scope of the prior sentence .09 .04 

Inappropriate justification .04 .02 

Saquentiais - Total number of correct responses 4 4 44 

Introduces new or next point .77 .75 

Temporal relation 0 0 

Sums up prior or previews subsequent information .20 .23 

Inappropriate justification 0 0 

l\/1iscellaneous (exclusion, guessing, restating text) .03 .03 

3See Appendix A in Goldman and Murray (1989) for a full description of the justification 
categories. 

in 
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Table 18 

Distribution of Justifications for incorrect Responses in Connector Experiment 1 

Type of incorrect Response Justification Appropriate to 

Being Justified Frequency Additive Causai Adversative Sequential IVIisc. 

Additive 

Native English 14 .14 .57 .07 .21 0 

ESL 23 0 .65 .13 .09 .13 

Causal 

Native English 9 .44 .22 .1 1 0 .22 

ESL 32 .44 .22 .09 .16 .09 

Adversative 

Native English 1 6 ,13 .56 .06 .25 0 

ESL 32 .34 .34 0 .13 .19 

Sequential 



Native English 1 7 .24 .35 0 .24 



II 



.18 



ESL 35 .34 .23 .1 1 .09 .23 



Table 19 



Mean Number Correct on Each Type of Connector and Mean Confidence Ratings for Correct 
Responses in Connector Experiment ?3 

Connector Mean 

Correct 

Additive 5.88 
Causal 6.66 
Adversative 5.63 
Sequential 5.22 

^The means are based on 32 subjects and the maximum score was 8 for correct 
responses. Confidence ratings are on a 7-point scale with 1 = very low confidence 
and 7 » very high confidence. 



Mean Confidence 
Rating for Correct 

5.54 

5.67 

5.98 

5.97 
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Table 20 

Distractof Selections and Mean Confidence Ratings in Connector Experiment 2 



Distractor 

Additive 
Causal 
Adversative 
Sequential 



Proportion of 
Incorrect Responses^ 
.27 
.42 
.13 
.16 



Mean Confidence 
Rating for incorrect^ 
5.04 
5.39 
5.07 
4.97 



^Tfie mean errors per subject was 8.62 out oi a maximum of 32. The Anova was conducted 
on tfie arcslne transformations of tfie proportion of incorrect responses. The means of the 
transformed proportions were as follows: additive «= 1.06; causal =1.44; adversative = 
.68; sequential - .77. 

•^The means for the confidence ratings were based on a 7-point scale with 1 = very low 
confidence and 7 = very high confidence. The means are based on 21 subjects because 1 1 did 
not have data in all 4 cells. 
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Table 21 

The Proportion of Errors on Each Connector Type Accounted for by Each of the 
Distractor Options in Connector Experiment 2^ 



Correct Connector 

Additive 
Causal 
Adversative 
Sequential 



Distractor Options 
Additive Causal Adversative 



.27 
.46 
.37 



.60 

.42 
.42 



.13 
.27 

.20 



Sequential 



.26 
.44 
,11 



^The proportions are based on the total number of errors for each connector type. The 
number of errors were the following: On additive slots, 68; on causal, 43; on adversative, 
76; and on the sequential, 89. 
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Table 22 

Student Selection Rate Dl'itributions for Connector Experiment 2 



Percentage 
Connector of Selections^ 



Additives 
Causals 
Adversative 
Sequential 



26 
32 
21 
21 



Percentage of Selections 
Scored as Correct 



70 
66 
82 
79 



Mean Confidence 
for Correct Responses 



5.54 
5.64 
5.96 
5.97 



^Total number of selectiofis was 1024. 
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Table 23 

Characteristics of the eSL Community College Students In Connector Experiment 3^ 

Characteristic Mean Range 

Age 27.8 19 - 65 

Age atwiilch school Ing In English began 20.6 3 - 60 

Stanford Diagnostic Reading Test, Grade Equivalent 5.30 2.7 - 10.6 

Social Science Courses In High School or College^ 2.34 o - 28 

Native l-anguage 

Spanish 48.5% 
European (except Spanish) 8.5% 
Asian 43.0% 
^Data are self-reported except for the Stanford Diagnostic Reading Test (SDRT) Brown, 
Form A. The SDRT Is administered by the community college for placement purposes when 
students enroll. Scores were available on all students. Only 2 students reported a TOEFL 
score 

^^Only 15 of the 35 students had taken courses related to Social Sciences in High School and 
only 9 had fallen such courses In community college. 
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Table 24 

Mean probability of a correct response, mean number correct, and mean confidence rating 
for correct items in Connector Experiment 3^ 





Mean 


Mean Number 


Mean Confidence 


Connector 


Probability 


Correct 


Ratings for Correct^' 


Additive 


.61 


4.87 


5.1 1 


Causal 


.53 


4.72 


4.92 


Adversative 


.46 


3.76 


4.99 


Sequential 


.39 


2.89 


5.21 



3The maximum number correct was 8 per connector type, except for the sequential 
connector. The maximum correct for the sequential connector was 7.46 because for 
19 of the students one of the sequential slots was discounted due to a typographical 
error in the passage. The error was corrected for the other 16 students. The data 
for probability and mean number correct are weighted appropriately. Due to this 
experimenter-error, the ANOVA was done on the probability correct scores. 
^The means are based on the 32 students who had at least one correct response for 
each of the connectors. Confidence ratings are based on a 7-point scale with 1 = very 
low confidence and 7 = very high confidence. 
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Table 25 



Distractor Selections and Mean Confidence Ratings in Connector Experiment 3 



Distractor 



Proportion of 
Incorrect Responses^ 



Mean Confidence 
Rating for Incorrect^ 



Additive 
Causal 
Adversative 
Sequential 



.25 
.39 
.20 
.13 



4.76 
4.63 
4.82 
4.50 



3The mean errors per subject was 15.77 out of a maximum of 32 The Anova was 
conducted on ths arcsine transformations of the proportion of incorrect responses. 
The means of the transformed proportions were as follows: additive = 1.05; causal 
» 1.36; adversative = .91; sequential = .72. 

^Three subjects did not nave data In all 4 cells and the means are based cn 32 subjects. 
Confidence ratings were based on a 7-point scale with 1 = very low confidence and 7 = very 
high confic'ence. 
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Tuble 26 

The Proportion of Errors on Each Connector Type Accounted for by Each of the Distractor 
Options for Connector Experiment 3^ 
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Correct Connector 



Additive 
Causal 
Adversative 
Sequential 



Additive 



.45 
.28 
.27 



Distractor Options 
Causai Adversative 



.50 

.44 
.53 



.39 
.37 

.19 



Sequential 



.10 
.17 
.26 



^The proportions are based on the total number of errors for each connector type. The 
number of errors were the following: On additive slots, 108; on causal, 132; on 
adversative, 152; and on the sequentials, 170. The number of possible errors was 280 for 
all but the sequential slots, which had a maximum of 261 . 
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Table 27 

Student Selection Rate Distributions for Connector Experiment 3 



Percentage Percentage of Seleciions 
Connector of Selections^ Scored as Correct 



Mean Confidence for 
Correct Responses 



Additives 
Causals 
Adversative 
Sequential 



29 
34 
23 
1 6 



53 
42 
51 
57 



5.27 
5.02 
5.09 
5.14 



^Total number of selections was 1098, representing 32 blanks for 16 subjects and 31 for 
19 subjects, less three instances of no response. 
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Table 28 

Distribution of Justifications for Correci Responses by ESL Community Coiiege Students in 



Connector Experiment 3 
Connector and Justification Category 

Additives - Totai number of correct responses 6 ' 

information gives exampie of concept .44 

information elaborates prior information .31 

inappropriate justification .11 

Causals ■ Total number of correct responses 4 8 

Cause - effect relationship between the two sentences .54 

Consistent but vaguely stated logical relationship .06 

Conclusion about a cause - effect relationship that was 

developed over several sentences. 0 

Inappropriate justification .15 

Adversatives - Total number of correct responses 4 7 

Comparison or contrast with information in ihe 

prior sentence; unexpected information .66 

Restriction of the scope of the prior sentence 0 

Inappropriate justification .19 

Sequentials ■ Total number of correct responses 3 4 

Introduces new or next point .26 

Temporal relation .26 

Sums up prior or previews subsequent information .12 

Inappropriate justification .03 

Miscellaneous^ (exclusion, guessing, restating text) .37 



^Miscellaneous did not differ by connector type and the .37 represents the rate of 
this response over all connector types. 
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1.1 




0.5 -4—- — I — < I — I — - ' ' I ' 1' ' ! 

Full Number Vague None 

Condition 

Figure 1: Mean reading rate per word and mean processing 

time in four signalling conditions for ESL speakers 
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0.8 



0.7- 




0.3 H 1 ' I 'I ' l l 

Full Numbor Vague None 

Condition 

Figure 2: Mean reading rate per word and mean processing^ 
time in four signaiiing conditions for Native Engiish speakers 
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SEG MARKER TIME/READING 0 



1 


!1 


8.183 


2 


12 


5.467 


3 


13 


7.183 


4 


TS 


5.083 


5 


F1 


5.533 


6 


P1.1 


5.033 


7 


PI .2 


8.460 


8 


P2 


12.517 


9 


P2.1 


4.933 


10 


P2.2 


3.200 


1 1 


P3 


4.167 


12 


P3.1 


5.083 


13 


P3.2 


5.900 


14 


P4 


12.300 


15 


P4.1 


8.717 


16 


P4.2 


5.467 


17 


F1 


19.533 


18 


F2 


9.717 


19 


F3 


8.400 



PASSAGE #13: t-attern Recognition of Speech Signals 



Figure 3: Once Through Strategy 



S83 MARKER TIME & READING ORDER 



1 


11 


9.267 


1 4.033 




2 


12 


6.783 


1 .983 




3 


13 


6.717 


1.817 




4 


TS 


4.450 


2.233 




5 


PI 


4.767 


1 .350 




6 


P1.1 


0.500 


1.133 




7 


PI. 2 


7.950 


1 .333 




8 


P2 


5.033 


;i.5i7 




9 


P2.1 


4.317 ' 


' 1.767 




10 


P2.2 


10.617 


8.117 




1 1 


P3 


4.067 


3.333 .6.450. 




12 


P3.1 


8,600 


5.333 / N 




13 


P3.2 


19.533 


1.967/ 


^-2.100 


14 


P4 


4.783 


6.800^ 


5.367 


15 


P4.1 


7.383 




13.367 


16 


P4.2 


7.550 




8.367 


17 


F1 


7.583 1 




2.817 


18 


F2 


7.750 /7.68i 




4.667 


19 


F3 


5.333^ 




3.300 




PASSAGE #8: Location 





Figure 4. Review Strategy 
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SEG 
1 
2 
3 
4 
5 
6 
7 
8 
9 

1 0 
1 1 
12 
1 3 
14 
1 5 
16 
17 
1 8 
1 9 



MARKER 
II 
12 
13 
TS 
PI 
P1.1 
P1.2 
P2 
P2.1 
P2.2 
P3 
P3.1 
P3.2 
P4 
P4.1 
P4.2 
F-1 
F2 
F3 



TIME & READING ORDER 
17.400 ^ 6.467 
2.317/^ 7.833 
12.817 
4.767 
9.500 
22.600 
3.833 




i1.250 



1 .083' 



4.167 
2.967 
1.200 
3.967 
1.617 
2.650 
2.267 
0.967 
1.483 
0.883 
2.050 
1.167 
1.133 
2.283 
2.533 



PASSAGE #16: The Nineteenth Century 



Figure 5. Regress Strategy 
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Figure 6: 



Once through 
Review 
Regress 




ESL Mono Eng 

Language Group 

Proportion of Passages Read with Each 
Type of Approach by Each Language Group 
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Figure 8: Local strategies for Review Strategy Passages 



Native English students 




Native English students 




■ Introduction 

Topic 
Q Target 
^ Elaboration 
D Conclusion 

Total • 83 



ESL students 




Native English students 




m Introduction 

0 Topic 

S Target 

0 Elaboration 

D Conclusion 



Total s 236 



ESL students 




■ Introduction 

Si Topic 

il Target 

0 Elaboration 

n Conclusion 



Total « 280 



Figure 11 
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Sentences that initiated direction changes in Regress Strategy 
passages before reaching the end 

133 



ESL Students 

Before reaching the end of passage 




I Introduction 

0 Topic Sentence 

H Target Points 

0 Elaborations 

D Conclusion 

Total - 91 



Fig 12. Sentences read to strategically in Regress Strategy passages 
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For incorrect cloze completions: 



Was the Justification appropriate 
to the choice? 




Yes 



Faulty text 
understanding 



Was the Justification appropriate 
to the cloze slot? 



Yes 
I 

Faulty connector 
understanding 



No 
I 

Faulty text and 
faulty connector 
understanding 



Figure 13: Relationships between incorrect cloze completions 
and verbal justifications 
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